2024-09-13T16:34:43.273 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T16:34:43.294 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T16:34:43.396 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832 branch: wip-rishabh-testing-20240910.095512-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable} email: null first_in_suite: false job_id: '7903832' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-rishabh-testing-20240910.095512-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 50bbe7f395997dcd871813949c49addbbf630361 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 50bbe7f395997dcd871813949c49addbbf630361 kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.095512-reef sha1: 50bbe7f395997dcd871813949c49addbbf630361 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1152 sha1: 50bbe7f395997dcd871813949c49addbbf630361 sleep_before_teardown: 0 subset: 984/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.095512-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 50bbe7f395997dcd871813949c49addbbf630361 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_openfiletable teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:36:41 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T16:34:43.396 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa; will attempt to use it 2024-09-13T16:34:43.397 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa/tasks 2024-09-13T16:34:43.397 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T16:34:43.399 INFO:teuthology.task.internal:Checking packages... 2024-09-13T16:34:43.420 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '50bbe7f395997dcd871813949c49addbbf630361' 2024-09-13T16:34:43.420 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T16:34:43.420 INFO:teuthology.packaging:ref: None 2024-09-13T16:34:43.420 INFO:teuthology.packaging:tag: None 2024-09-13T16:34:43.421 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:34:43.421 INFO:teuthology.packaging:sha1: 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:34:43.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:34:43.585 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:34:43.587 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T16:34:43.593 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T16:34:43.593 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T16:34:43.605 INFO:teuthology.task.internal:Saving configuration 2024-09-13T16:34:43.622 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T16:34:43.628 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T16:34:43.663 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 16:28:07.255458', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T16:34:43.697 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 16:28:07.257533', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T16:34:43.697 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T16:34:43.702 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-13T16:34:43.702 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-13T16:34:43.702 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T16:34:43.776 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f760875fbe0>, signals=[15]) 2024-09-13T16:34:43.777 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T16:34:43.782 INFO:teuthology.task.internal:Opening connections... 2024-09-13T16:34:43.783 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:34:43.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:34:43.901 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:34:43.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:34:44.002 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T16:34:44.008 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-09-13T16:34:44.015 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-09-13T16:34:44.015 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Ubuntu" 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:ID=ubuntu 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE=debian 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="20.04" 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T16:34:44.062 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_CODENAME=focal 2024-09-13T16:34:44.063 INFO:teuthology.orchestra.run.smithi003.stdout:UBUNTU_CODENAME=focal 2024-09-13T16:34:44.063 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-09-13T16:34:44.100 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-09-13T16:34:44.107 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-09-13T16:34:44.107 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Ubuntu" 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:ID=ubuntu 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE=debian 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="20.04" 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T16:34:44.156 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T16:34:44.157 INFO:teuthology.orchestra.run.smithi157.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T16:34:44.157 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_CODENAME=focal 2024-09-13T16:34:44.157 INFO:teuthology.orchestra.run.smithi157.stdout:UBUNTU_CODENAME=focal 2024-09-13T16:34:44.157 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-09-13T16:34:44.192 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T16:34:44.201 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T16:34:44.207 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T16:34:44.207 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-09-13T16:34:44.209 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-09-13T16:34:44.214 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T16:34:44.220 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T16:34:44.220 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-09-13T16:34:44.258 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-09-13T16:34:44.274 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T16:34:44.343 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T16:34:44.357 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-13T16:34:44.357 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-13T16:34:44.357 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-13T16:34:44.358 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T16:34:44.358 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T16:34:44.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:34:44.363 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2024-09-13T16:34:44.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:34:44.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2024-09-13T16:34:44.540 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T16:34:44.540 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-13T16:34:44.541 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-13T16:34:44.547 INFO:teuthology.orchestra.run.smithi003.stdout:5.4.0-124-generic 2024-09-13T16:34:44.547 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T16:34:44.547 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-13T16:34:44.548 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-13T16:34:44.548 DEBUG:teuthology.orchestra.run.smithi003:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-13T16:34:44.560 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T16:34:44.560 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-13T16:34:44.560 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-13T16:34:44.566 INFO:teuthology.orchestra.run.smithi157.stdout:5.4.0-124-generic 2024-09-13T16:34:44.566 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T16:34:44.567 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-13T16:34:44.567 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-13T16:34:44.567 DEBUG:teuthology.orchestra.run.smithi157:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-13T16:34:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2024-09-13T16:34:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2024-09-13T16:34:44.672 INFO:teuthology.orchestra.run.smithi003.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-13T16:34:44.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-13T16:34:44.694 INFO:teuthology.orchestra.run.smithi157.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-13T16:34:44.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-13T16:34:44.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-13T16:34:44.717 DEBUG:teuthology.orchestra.run.smithi003:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-13T16:34:44.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-13T16:34:44.782 DEBUG:teuthology.orchestra.run.smithi157:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-13T16:35:16.383 INFO:teuthology.orchestra.run.smithi157.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-13T16:35:16.384 INFO:teuthology.orchestra.run.smithi157.stderr: * [new branch] main -> origin/main 2024-09-13T16:35:16.449 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190312 -> 20190312 2024-09-13T16:35:16.449 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190416 -> 20190416 2024-09-13T16:35:16.449 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190514 -> 20190514 2024-09-13T16:35:16.449 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190618 -> 20190618 2024-09-13T16:35:16.450 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190717 -> 20190717 2024-09-13T16:35:16.450 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190815 -> 20190815 2024-09-13T16:35:16.450 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190923 -> 20190923 2024-09-13T16:35:16.450 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20191022 -> 20191022 2024-09-13T16:35:16.450 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20191215 -> 20191215 2024-09-13T16:35:16.451 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200122 -> 20200122 2024-09-13T16:35:16.451 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200316 -> 20200316 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200421 -> 20200421 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200519 -> 20200519 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200619 -> 20200619 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200721 -> 20200721 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200817 -> 20200817 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200918 -> 20200918 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20201022 -> 20201022 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20201118 -> 20201118 2024-09-13T16:35:16.452 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20201218 -> 20201218 2024-09-13T16:35:16.453 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210208 -> 20210208 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210315 -> 20210315 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210511 -> 20210511 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210716 -> 20210716 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210818 -> 20210818 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210919 -> 20210919 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20211027 -> 20211027 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20211216 -> 20211216 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220209 -> 20220209 2024-09-13T16:35:16.454 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220310 -> 20220310 2024-09-13T16:35:16.455 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220411 -> 20220411 2024-09-13T16:35:16.455 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220509 -> 20220509 2024-09-13T16:35:16.455 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220610 -> 20220610 2024-09-13T16:35:16.457 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220708 -> 20220708 2024-09-13T16:35:16.457 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220815 -> 20220815 2024-09-13T16:35:16.457 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220913 -> 20220913 2024-09-13T16:35:16.457 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20221012 -> 20221012 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20221109 -> 20221109 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20221214 -> 20221214 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230117 -> 20230117 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230210 -> 20230210 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230310 -> 20230310 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230404 -> 20230404 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230515 -> 20230515 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230625 -> 20230625 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230804 -> 20230804 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230919 -> 20230919 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20231030 -> 20231030 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20231111 -> 20231111 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20231211 -> 20231211 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240115 -> 20240115 2024-09-13T16:35:16.458 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240220 -> 20240220 2024-09-13T16:35:16.459 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240312 -> 20240312 2024-09-13T16:35:16.459 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240410 -> 20240410 2024-09-13T16:35:16.459 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240513 -> 20240513 2024-09-13T16:35:16.459 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240610 -> 20240610 2024-09-13T16:35:16.459 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240709 -> 20240709 2024-09-13T16:35:16.459 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240811 -> 20240811 2024-09-13T16:35:16.459 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240909 -> 20240909 2024-09-13T16:35:18.219 INFO:teuthology.orchestra.run.smithi003.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-13T16:35:18.220 INFO:teuthology.orchestra.run.smithi003.stderr: * [new branch] main -> origin/main 2024-09-13T16:35:18.290 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190312 -> 20190312 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190416 -> 20190416 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190514 -> 20190514 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190618 -> 20190618 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190717 -> 20190717 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190815 -> 20190815 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190923 -> 20190923 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20191022 -> 20191022 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20191215 -> 20191215 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200122 -> 20200122 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200316 -> 20200316 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200421 -> 20200421 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200519 -> 20200519 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200619 -> 20200619 2024-09-13T16:35:18.291 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200721 -> 20200721 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200817 -> 20200817 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200918 -> 20200918 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20201022 -> 20201022 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20201118 -> 20201118 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20201218 -> 20201218 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210208 -> 20210208 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210315 -> 20210315 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210511 -> 20210511 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210716 -> 20210716 2024-09-13T16:35:18.292 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210818 -> 20210818 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210919 -> 20210919 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20211027 -> 20211027 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20211216 -> 20211216 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220209 -> 20220209 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220310 -> 20220310 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220411 -> 20220411 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220509 -> 20220509 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220610 -> 20220610 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220708 -> 20220708 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220815 -> 20220815 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220913 -> 20220913 2024-09-13T16:35:18.296 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20221012 -> 20221012 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20221109 -> 20221109 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20221214 -> 20221214 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230117 -> 20230117 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230210 -> 20230210 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230310 -> 20230310 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230404 -> 20230404 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230515 -> 20230515 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230625 -> 20230625 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230804 -> 20230804 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230919 -> 20230919 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20231030 -> 20231030 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20231111 -> 20231111 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20231211 -> 20231211 2024-09-13T16:35:18.297 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240115 -> 20240115 2024-09-13T16:35:18.298 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240220 -> 20240220 2024-09-13T16:35:18.298 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240312 -> 20240312 2024-09-13T16:35:18.298 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240410 -> 20240410 2024-09-13T16:35:18.298 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240513 -> 20240513 2024-09-13T16:35:18.298 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240610 -> 20240610 2024-09-13T16:35:18.298 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240709 -> 20240709 2024-09-13T16:35:18.298 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240811 -> 20240811 2024-09-13T16:35:18.298 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20240909 -> 20240909 2024-09-13T16:35:21.001 INFO:teuthology.orchestra.run.smithi157.stderr:Updating files: 44% (1582/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 59% (2077/3520) Updating files: 59% (2084/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 69% (2429/3520) Updating files: 69% (2440/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 82% (2887/3520) Updating files: 82% (2908/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-13T16:35:21.006 INFO:teuthology.orchestra.run.smithi157.stdout:HEAD is now at a34e7a5f Merge branch 'fix-rawfile-links' into 'main' 2024-09-13T16:35:21.024 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-13T16:35:21.024 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T16:35:21.180 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/focal/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T16:35:21.180 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/focal/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-13T16:35:21.185 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi157.front.sepia.ceph.com to complete... 2024-09-13T16:35:23.100 INFO:teuthology.orchestra.run.smithi003.stderr:Updating files: 29% (1031/3520) Updating files: 30% (1056/3520) Updating files: 31% (1092/3520) Updating files: 32% (1127/3520) Updating files: 33% (1162/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2060/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 69% (2429/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 82% (2887/3520) Updating files: 82% (2895/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-13T16:35:23.105 INFO:teuthology.orchestra.run.smithi003.stdout:HEAD is now at a34e7a5f Merge branch 'fix-rawfile-links' into 'main' 2024-09-13T16:35:23.124 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-13T16:35:23.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T16:35:23.279 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/focal/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T16:35:23.279 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/focal/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-13T16:35:23.281 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi003.front.sepia.ceph.com to complete... 2024-09-13T16:35:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13 16:35:23 URL:https://2.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/focal/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45355934/45355934] -> "/tmp/linux-image.deb" [1] 2024-09-13T16:35:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:FINISHED --2024-09-13 16:35:23-- 2024-09-13T16:35:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:Total wall clock time: 1.1s 2024-09-13T16:35:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:Downloaded: 1 files, 43M in 0.9s (46.2 MB/s) 2024-09-13T16:35:24.373 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-13T16:35:24.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo dpkg -i /tmp/linux-image.deb 2024-09-13T16:35:24.647 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-13T16:35:25.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13 16:35:25 URL:https://2.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/focal/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45355934/45355934] -> "/tmp/linux-image.deb" [1] 2024-09-13T16:35:25.384 INFO:teuthology.orchestra.run.smithi157.stderr:FINISHED --2024-09-13 16:35:25-- 2024-09-13T16:35:25.384 INFO:teuthology.orchestra.run.smithi157.stderr:Total wall clock time: 4.1s 2024-09-13T16:35:25.384 INFO:teuthology.orchestra.run.smithi157.stderr:Downloaded: 1 files, 43M in 4.0s (10.8 MB/s) 2024-09-13T16:35:25.386 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-13T16:35:25.387 DEBUG:teuthology.orchestra.run.smithi157:> sudo dpkg -i /tmp/linux-image.deb 2024-09-13T16:35:25.714 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-13T16:35:26.376 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... 104373 files and directories currently installed.) 2024-09-13T16:35:26.381 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-13T16:35:26.511 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-13T16:35:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... 104373 files and directories currently installed.) 2024-09-13T16:35:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-13T16:35:27.652 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-13T16:35:29.997 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-13T16:35:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:35:31.107 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-13T16:35:31.220 INFO:teuthology.orchestra.run.smithi157.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:35:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub' 2024-09-13T16:35:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T16:35:58.554 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-09-13T16:35:58.920 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:35:58.944 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:35:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T16:35:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T16:35:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub' 2024-09-13T16:35:59.661 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T16:35:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-09-13T16:36:00.040 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:00.066 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:00.485 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T16:36:00.508 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T16:36:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-09-13T16:36:01.670 DEBUG:teuthology.orchestra.run.smithi003:> dpkg-deb -c /tmp/linux-image.deb 2024-09-13T16:36:01.678 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./ 2024-09-13T16:36:01.678 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./boot/ 2024-09-13T16:36:01.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:27 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:01.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:27 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:01.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:27 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/postinst.d/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/postrm.d/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/preinst.d/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/prerm.d/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-13T16:36:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-13T16:36:01.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-13T16:36:01.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-13T16:36:01.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-13T16:36:01.731 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-13T16:36:01.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-13T16:36:01.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-13T16:36:01.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-13T16:36:01.781 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-13T16:36:01.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-13T16:36:01.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-13T16:36:01.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-13T16:36:01.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-13T16:36:01.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-13T16:36:01.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-13T16:36:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-13T16:36:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-13T16:36:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-13T16:36:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-13T16:36:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-13T16:36:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-13T16:36:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-13T16:36:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-13T16:36:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-13T16:36:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-13T16:36:01.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-13T16:36:01.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-13T16:36:01.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-13T16:36:01.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-13T16:36:01.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-13T16:36:01.799 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-13T16:36:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-13T16:36:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-13T16:36:01.802 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-13T16:36:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-13T16:36:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-13T16:36:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-13T16:36:01.806 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-13T16:36:01.807 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-13T16:36:01.807 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-13T16:36:01.807 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-13T16:36:01.807 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-13T16:36:01.807 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-13T16:36:01.807 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-13T16:36:01.807 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-13T16:36:01.808 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-13T16:36:01.809 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-13T16:36:01.810 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-13T16:36:01.811 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-13T16:36:01.812 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-13T16:36:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-13T16:36:01.814 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-13T16:36:01.815 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-13T16:36:01.816 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-13T16:36:01.817 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-13T16:36:01.818 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-13T16:36:01.819 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-13T16:36:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-13T16:36:01.821 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-13T16:36:01.822 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-13T16:36:01.823 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-13T16:36:01.824 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-13T16:36:01.847 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-13T16:36:01.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-13T16:36:01.847 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-13T16:36:01.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-13T16:36:01.847 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-13T16:36:01.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-13T16:36:01.847 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-13T16:36:01.847 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-13T16:36:01.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-13T16:36:01.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-13T16:36:01.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-13T16:36:01.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-13T16:36:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-13T16:36:01.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-13T16:36:01.854 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-13T16:36:01.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-13T16:36:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-13T16:36:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-13T16:36:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-13T16:36:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-13T16:36:01.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-13T16:36:01.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-13T16:36:01.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-13T16:36:01.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-13T16:36:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-13T16:36:01.865 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-13T16:36:01.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-13T16:36:01.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-13T16:36:01.868 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-13T16:36:01.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-13T16:36:01.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-13T16:36:01.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-13T16:36:01.872 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-13T16:36:01.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-13T16:36:01.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-13T16:36:01.875 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-13T16:36:01.876 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-13T16:36:01.876 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-13T16:36:01.876 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-13T16:36:01.876 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-13T16:36:01.899 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-13T16:36:01.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-13T16:36:01.899 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-13T16:36:01.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-13T16:36:01.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-13T16:36:01.899 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-13T16:36:01.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-13T16:36:01.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-13T16:36:01.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-13T16:36:01.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-13T16:36:01.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-13T16:36:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-13T16:36:01.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-13T16:36:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-13T16:36:01.907 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-13T16:36:01.907 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-13T16:36:01.907 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-13T16:36:01.907 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-13T16:36:01.907 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-13T16:36:01.907 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-13T16:36:01.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-13T16:36:01.909 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-13T16:36:01.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-13T16:36:01.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-13T16:36:01.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-13T16:36:01.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-13T16:36:01.914 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-13T16:36:01.915 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-13T16:36:01.916 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-13T16:36:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-13T16:36:01.918 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-13T16:36:01.919 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-13T16:36:01.920 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-13T16:36:01.921 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-13T16:36:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-13T16:36:01.923 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-13T16:36:01.924 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-13T16:36:01.925 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-13T16:36:01.925 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-13T16:36:01.925 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-13T16:36:01.925 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-13T16:36:01.925 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-13T16:36:01.925 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-13T16:36:01.958 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-13T16:36:01.959 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-13T16:36:01.959 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-13T16:36:01.959 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-13T16:36:01.959 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-13T16:36:01.959 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-13T16:36:01.959 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-13T16:36:01.959 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-13T16:36:01.960 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-13T16:36:01.961 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-13T16:36:01.962 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-13T16:36:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-13T16:36:01.964 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-13T16:36:01.965 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-13T16:36:01.966 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-13T16:36:01.967 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-13T16:36:01.968 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-13T16:36:01.969 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-13T16:36:01.970 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-13T16:36:01.971 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-13T16:36:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-13T16:36:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-13T16:36:01.974 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-13T16:36:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-13T16:36:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-13T16:36:01.977 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-13T16:36:01.978 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-13T16:36:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-13T16:36:01.980 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-13T16:36:01.981 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-13T16:36:01.982 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-13T16:36:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-13T16:36:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-13T16:36:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-13T16:36:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-13T16:36:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-13T16:36:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-13T16:36:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-13T16:36:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-13T16:36:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-13T16:36:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-13T16:36:01.997 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-13T16:36:01.997 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-13T16:36:01.997 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-13T16:36:01.998 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-13T16:36:01.999 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-13T16:36:02.000 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-13T16:36:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-13T16:36:02.002 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-13T16:36:02.003 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-13T16:36:02.004 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-13T16:36:02.005 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-13T16:36:02.006 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-13T16:36:02.007 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-13T16:36:02.007 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-13T16:36:02.007 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-13T16:36:02.007 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-13T16:36:02.007 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-13T16:36:02.007 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-13T16:36:02.007 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-13T16:36:02.008 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-13T16:36:02.008 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-13T16:36:02.008 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-13T16:36:02.009 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-13T16:36:02.009 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-13T16:36:02.009 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-13T16:36:02.009 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-13T16:36:02.009 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-13T16:36:02.009 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-13T16:36:02.009 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-13T16:36:02.009 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-13T16:36:02.010 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-13T16:36:02.017 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-13T16:36:02.017 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-13T16:36:02.017 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-13T16:36:02.017 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-13T16:36:02.017 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-13T16:36:02.018 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-13T16:36:02.019 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-13T16:36:02.022 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-13T16:36:02.023 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-13T16:36:02.024 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-13T16:36:02.025 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-13T16:36:02.025 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-13T16:36:02.026 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-13T16:36:02.026 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-13T16:36:02.026 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-13T16:36:02.026 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-13T16:36:02.026 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-13T16:36:02.026 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-13T16:36:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-13T16:36:02.028 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-13T16:36:02.028 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-13T16:36:02.028 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-13T16:36:02.028 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-13T16:36:02.028 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-13T16:36:02.028 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-13T16:36:02.029 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-13T16:36:02.030 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-13T16:36:02.030 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-13T16:36:02.030 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-13T16:36:02.030 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-13T16:36:02.030 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-13T16:36:02.031 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-13T16:36:02.031 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-13T16:36:02.031 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-13T16:36:02.031 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-13T16:36:02.031 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-13T16:36:02.038 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-13T16:36:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-13T16:36:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-13T16:36:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-13T16:36:02.042 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-13T16:36:02.043 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-13T16:36:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-13T16:36:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-13T16:36:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-13T16:36:02.047 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-13T16:36:02.048 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-13T16:36:02.048 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-13T16:36:02.048 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-13T16:36:02.048 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-13T16:36:02.048 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-13T16:36:02.062 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-13T16:36:02.062 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-13T16:36:02.062 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-13T16:36:02.063 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-13T16:36:02.064 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-13T16:36:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-13T16:36:02.066 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-13T16:36:02.067 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-13T16:36:02.068 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-13T16:36:02.069 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-13T16:36:02.070 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-13T16:36:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-13T16:36:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-13T16:36:02.073 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-13T16:36:02.074 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-13T16:36:02.075 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-13T16:36:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-13T16:36:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-13T16:36:02.078 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-13T16:36:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-13T16:36:02.082 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 210 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 879 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./usr/ 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./usr/share/ 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./usr/share/doc/ 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 175 2024-09-06 14:27 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-13T16:36:02.083 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 691 2024-09-06 14:27 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-13T16:36:02.088 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:02.088 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:02.088 DEBUG:teuthology.orchestra.run.smithi003:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-13T16:36:02.133 INFO:teuthology.orchestra.run.smithi003.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2024-09-13T16:36:02.133 INFO:teuthology.orchestra.run.smithi003.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2024-09-13T16:36:02.133 INFO:teuthology.orchestra.run.smithi003.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2024-09-13T16:36:02.134 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-13T16:36:02.134 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:02.134 DEBUG:teuthology.orchestra.run.smithi003:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-13T16:36:02.136 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi003.front.sepia.ceph.com to complete... 2024-09-13T16:36:02.721 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-09-13T16:36:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub' 2024-09-13T16:36:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T16:36:02.853 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-09-13T16:36:02.904 DEBUG:teuthology.orchestra.run.smithi157:> dpkg-deb -c /tmp/linux-image.deb 2024-09-13T16:36:02.913 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./ 2024-09-13T16:36:02.913 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./boot/ 2024-09-13T16:36:02.914 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:27 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:02.936 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:27 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:02.937 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:27 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:02.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/ 2024-09-13T16:36:02.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/ 2024-09-13T16:36:02.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/postinst.d/ 2024-09-13T16:36:02.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/postrm.d/ 2024-09-13T16:36:02.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/preinst.d/ 2024-09-13T16:36:02.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./etc/kernel/prerm.d/ 2024-09-13T16:36:02.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/ 2024-09-13T16:36:02.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/ 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-13T16:36:02.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-13T16:36:02.963 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-13T16:36:02.964 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-13T16:36:02.964 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-13T16:36:02.964 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-13T16:36:02.964 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-13T16:36:02.964 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-13T16:36:02.964 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-13T16:36:02.964 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-13T16:36:02.965 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-13T16:36:02.965 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-13T16:36:02.965 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-13T16:36:02.965 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-13T16:36:02.965 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-13T16:36:02.981 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-13T16:36:02.981 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-13T16:36:02.981 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-13T16:36:02.981 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-13T16:36:02.981 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-13T16:36:02.981 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-13T16:36:02.982 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-13T16:36:02.983 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-13T16:36:02.984 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-13T16:36:02.985 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-13T16:36:02.986 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-13T16:36:02.987 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-13T16:36:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-13T16:36:02.989 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-13T16:36:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-13T16:36:02.991 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-13T16:36:02.992 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-13T16:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-13T16:36:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-13T16:36:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-13T16:36:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-13T16:36:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-13T16:36:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-13T16:36:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-13T16:36:02.994 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-13T16:36:03.046 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-13T16:36:03.047 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-13T16:36:03.048 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-13T16:36:03.049 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-13T16:36:03.050 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-13T16:36:03.051 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-13T16:36:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-13T16:36:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-13T16:36:03.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-13T16:36:03.055 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-13T16:36:03.056 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-13T16:36:03.057 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-13T16:36:03.058 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-13T16:36:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-13T16:36:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-13T16:36:03.061 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-13T16:36:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-13T16:36:03.063 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-13T16:36:03.064 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-13T16:36:03.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-13T16:36:03.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-13T16:36:03.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-13T16:36:03.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-13T16:36:03.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-13T16:36:03.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-13T16:36:03.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-13T16:36:03.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-13T16:36:03.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-13T16:36:03.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-13T16:36:03.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-13T16:36:03.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-13T16:36:03.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-13T16:36:03.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-13T16:36:03.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-13T16:36:03.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-13T16:36:03.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-13T16:36:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-13T16:36:03.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-13T16:36:03.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-13T16:36:03.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-13T16:36:03.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-13T16:36:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-13T16:36:03.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-13T16:36:03.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-13T16:36:03.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-13T16:36:03.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-13T16:36:03.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-13T16:36:03.093 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-13T16:36:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-13T16:36:03.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-13T16:36:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-13T16:36:03.097 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-13T16:36:03.098 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-13T16:36:03.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-13T16:36:03.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-13T16:36:03.099 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-13T16:36:03.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-13T16:36:03.100 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:03.100 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:03.131 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-13T16:36:03.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-13T16:36:03.133 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-13T16:36:03.134 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-13T16:36:03.135 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-13T16:36:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-13T16:36:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-13T16:36:03.138 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-13T16:36:03.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-13T16:36:03.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-13T16:36:03.141 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-13T16:36:03.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-13T16:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-13T16:36:03.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-13T16:36:03.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-13T16:36:03.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-13T16:36:03.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-13T16:36:03.148 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-13T16:36:03.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-13T16:36:03.150 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-13T16:36:03.151 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-13T16:36:03.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-13T16:36:03.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-13T16:36:03.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-13T16:36:03.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-13T16:36:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-13T16:36:03.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-13T16:36:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-13T16:36:03.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-13T16:36:03.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-13T16:36:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-13T16:36:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-13T16:36:03.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-13T16:36:03.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-13T16:36:03.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-13T16:36:03.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-13T16:36:03.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-13T16:36:03.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-13T16:36:03.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-13T16:36:03.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-13T16:36:03.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-13T16:36:03.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-13T16:36:03.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-13T16:36:03.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-13T16:36:03.200 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-13T16:36:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-13T16:36:03.202 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-13T16:36:03.203 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-13T16:36:03.204 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-13T16:36:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-13T16:36:03.206 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-13T16:36:03.207 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-13T16:36:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-13T16:36:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-13T16:36:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-13T16:36:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-13T16:36:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-13T16:36:03.213 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-13T16:36:03.214 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-13T16:36:03.215 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-13T16:36:03.216 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-13T16:36:03.217 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-13T16:36:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-13T16:36:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-13T16:36:03.220 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-13T16:36:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-13T16:36:03.222 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-13T16:36:03.223 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-13T16:36:03.224 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-13T16:36:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-13T16:36:03.226 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-13T16:36:03.227 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-13T16:36:03.228 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-13T16:36:03.229 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-13T16:36:03.230 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-13T16:36:03.231 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-13T16:36:03.232 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-13T16:36:03.233 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-13T16:36:03.234 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-13T16:36:03.235 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-13T16:36:03.243 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-13T16:36:03.243 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-13T16:36:03.243 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-13T16:36:03.243 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-13T16:36:03.243 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-13T16:36:03.243 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-13T16:36:03.244 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-13T16:36:03.245 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-13T16:36:03.246 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-13T16:36:03.247 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-13T16:36:03.248 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-13T16:36:03.249 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-13T16:36:03.250 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-13T16:36:03.251 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-13T16:36:03.252 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-13T16:36:03.252 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-13T16:36:03.252 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-13T16:36:03.252 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-13T16:36:03.252 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-13T16:36:03.252 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-13T16:36:03.258 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-13T16:36:03.258 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-13T16:36:03.259 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-13T16:36:03.260 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-13T16:36:03.261 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-13T16:36:03.262 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-13T16:36:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-13T16:36:03.264 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-13T16:36:03.265 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-13T16:36:03.266 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-13T16:36:03.266 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-13T16:36:03.290 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-13T16:36:03.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-13T16:36:03.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-13T16:36:03.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-13T16:36:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-13T16:36:03.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-13T16:36:03.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-13T16:36:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-13T16:36:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-13T16:36:03.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-13T16:36:03.300 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-13T16:36:03.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-13T16:36:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-13T16:36:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-13T16:36:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-13T16:36:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-13T16:36:03.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-13T16:36:03.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-13T16:36:03.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-13T16:36:03.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-13T16:36:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-13T16:36:03.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-13T16:36:03.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-13T16:36:03.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-13T16:36:03.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-13T16:36:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-13T16:36:03.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-13T16:36:03.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-13T16:36:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-13T16:36:03.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-13T16:36:03.321 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-13T16:36:03.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-13T16:36:03.321 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-13T16:36:03.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 210 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-13T16:36:03.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 879 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:27 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./usr/ 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./usr/share/ 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./usr/share/doc/ 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:27 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 175 2024-09-06 14:27 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-13T16:36:03.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 691 2024-09-06 14:27 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-13T16:36:03.329 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:03.329 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:03.329 DEBUG:teuthology.orchestra.run.smithi157:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-13T16:36:03.375 INFO:teuthology.orchestra.run.smithi157.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2024-09-13T16:36:03.376 INFO:teuthology.orchestra.run.smithi157.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2024-09-13T16:36:03.376 INFO:teuthology.orchestra.run.smithi157.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2024-09-13T16:36:03.376 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-13T16:36:03.376 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:03.376 DEBUG:teuthology.orchestra.run.smithi157:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-13T16:36:03.379 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi157.front.sepia.ceph.com to complete... 2024-09-13T16:36:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T16:36:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T16:36:04.068 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub' 2024-09-13T16:36:04.079 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T16:36:04.091 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-09-13T16:36:04.315 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:04.327 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:36:04.530 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-09-13T16:36:04.726 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T16:36:04.738 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T16:36:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-09-13T16:36:35.665 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-13T16:36:35.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-13T16:36:35.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:36:36.803 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-13T16:36:36.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T16:36:36.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:37:11.361 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-13T16:37:20.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T16:37:20.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:37:23.453 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-13T16:37:35.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T16:37:35.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:37:35.677 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T16:37:44.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-13T16:37:44.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:37:44.697 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-09-13T16:37:56.709 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-13T16:37:56.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:37:56.977 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-13T16:37:57.668 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-13T16:37:57.668 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-13T16:37:57.669 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-13T16:37:57.727 INFO:teuthology.orchestra.run.smithi003.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:37:57.728 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:37:57.728 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T16:37:57.728 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T16:37:58.729 DEBUG:teuthology.parallel:result is None 2024-09-13T16:38:06.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-13T16:38:21.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T16:38:21.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:38:22.163 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-13T16:38:22.886 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T16:38:22.886 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-13T16:38:22.887 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-13T16:38:22.944 INFO:teuthology.orchestra.run.smithi157.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:38:22.945 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T16:38:22.945 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T16:38:22.945 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T16:38:23.947 DEBUG:teuthology.parallel:result is None 2024-09-13T16:38:23.947 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T16:38:23.954 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T16:38:23.954 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T16:38:23.960 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T16:38:23.976 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T16:38:23.982 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T16:38:23.989 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T16:38:23.989 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T16:38:24.022 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T16:38:24.057 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T16:38:24.065 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T16:38:24.066 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T16:38:24.090 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T16:38:24.200 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:38:24.213 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:38:24.225 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:38:24.243 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:38:24.247 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T16:38:24.254 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T16:38:24.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T16:38:24.260 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T16:38:24.295 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T16:38:24.304 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T16:38:24.304 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T16:38:24.341 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T16:38:24.357 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:38:24.417 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:38:24.478 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:38:24.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T16:38:24.628 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:38:24.645 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:38:24.703 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:38:24.703 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T16:38:24.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-09-13T16:38:24.792 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-09-13T16:38:24.939 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T16:38:24.946 INFO:teuthology.task.internal:Starting timer... 2024-09-13T16:38:24.946 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T16:38:24.956 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T16:38:24.964 DEBUG:teuthology.task.selinux:Excluding smithi003: OS 'ubuntu' does not support SELinux 2024-09-13T16:38:24.964 DEBUG:teuthology.task.selinux:Excluding smithi157: OS 'ubuntu' does not support SELinux 2024-09-13T16:38:24.965 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T16:38:24.965 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T16:38:24.965 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T16:38:24.965 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T16:38:24.973 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T16:38:24.984 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T16:38:24.985 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T16:45:46.163 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-09-13T16:45:46.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-13T16:45:46.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:45:46.282 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-13T16:45:46.525 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-13T16:45:46.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T16:45:46.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:45:46.639 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-13T16:45:46.848 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T16:45:46.849 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T16:45:46.858 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T16:45:46.858 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T16:45:46.859 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:45:46.865 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T16:45:46.865 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:45:46.937 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T16:45:46.937 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: Command line: ntpd -gq 2024-09-13T16:45:46.938 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: proto: precision = 0.061 usec (-24) 2024-09-13T16:45:46.939 INFO:teuthology.orchestra.run.smithi003.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T16:45:46.939 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T16:45:46.939 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T16:45:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T16:45:46.941 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T16:45:46.941 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T16:45:46.941 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T16:45:46.941 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: Listen normally on 3 ens1f0 172.21.15.3:123 2024-09-13T16:45:46.941 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: Listen normally on 4 lo [::1]:123 2024-09-13T16:45:46.941 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15dc%4]:123 2024-09-13T16:45:46.941 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:46 ntpd[13526]: Listening on routing socket on fd #22 for interface updates 2024-09-13T16:45:46.963 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T16:45:46.963 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: Command line: ntpd -gq 2024-09-13T16:45:46.964 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: proto: precision = 0.070 usec (-24) 2024-09-13T16:45:46.965 INFO:teuthology.orchestra.run.smithi157.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T16:45:46.965 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T16:45:46.965 INFO:teuthology.orchestra.run.smithi157.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T16:45:46.966 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T16:45:46.966 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T16:45:46.966 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T16:45:46.967 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T16:45:46.967 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: Listen normally on 3 enp3s0f1 172.21.15.157:123 2024-09-13T16:45:46.967 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: Listen normally on 4 lo [::1]:123 2024-09-13T16:45:46.967 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ea1%5]:123 2024-09-13T16:45:46.967 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:46 ntpd[13607]: Listening on routing socket on fd #22 for interface updates 2024-09-13T16:45:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:47 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:47 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:47 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:47 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:47 ntpd[13526]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:47 ntpd[13526]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:47.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:47.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:47.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:47 ntpd[13607]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:48 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:48 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:48 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:48 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:48 ntpd[13526]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:48 ntpd[13526]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:48.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:48 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:48.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:48 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:49.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:49 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:49.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:49 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:49.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:49 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:49.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:49 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:49.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:49 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:49.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:49 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:49.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:49 ntpd[13526]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:49.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:49 ntpd[13526]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:49.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:49.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:49 ntpd[13607]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:50.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:50 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:50.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:50 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:50.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:50 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:50.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:50 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:50.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:50 ntpd[13526]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:50.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:50 ntpd[13526]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:51.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.940 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:51.940 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:51.940 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.940 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.940 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:51.940 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:51 ntpd[13526]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:51 ntpd[13607]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:52 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:52 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:52 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:52 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:52 ntpd[13526]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:52 ntpd[13526]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:53.940 INFO:teuthology.orchestra.run.smithi003.stdout:13 Sep 16:45:53 ntpd[13526]: ntpd: time slew +0.001382 s 2024-09-13T16:45:53.940 INFO:teuthology.orchestra.run.smithi003.stdout:ntpd: time slew +0.001382s 2024-09-13T16:45:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T16:45:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:13 Sep 16:45:53 ntpd[13526]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T16:45:53.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:53.964 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:53.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:53.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:53.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:53.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:53.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:45:53.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:45:53.965 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:45:53.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:45:53.966 INFO:teuthology.orchestra.run.smithi157.stdout:13 Sep 16:45:53 ntpd[13607]: ntpd: time slew +0.001129 s 2024-09-13T16:45:53.966 INFO:teuthology.orchestra.run.smithi157.stdout:ntpd: time slew +0.001129s 2024-09-13T16:45:53.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T16:45:53.966 INFO:teuthology.orchestra.run.smithi157.stderr:13 Sep 16:45:53 ntpd[13607]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T16:45:54.100 INFO:teuthology.orchestra.run.smithi003.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T16:45:54.100 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================== 2024-09-13T16:45:54.100 INFO:teuthology.orchestra.run.smithi003.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:45:54.100 INFO:teuthology.orchestra.run.smithi003.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:45:54.100 INFO:teuthology.orchestra.run.smithi003.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:45:54.100 INFO:teuthology.orchestra.run.smithi003.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:45:54.137 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T16:45:54.137 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2024-09-13T16:45:54.137 INFO:teuthology.orchestra.run.smithi157.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:45:54.137 INFO:teuthology.orchestra.run.smithi157.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:45:54.137 INFO:teuthology.orchestra.run.smithi157.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:45:54.137 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:45:54.138 INFO:teuthology.run_tasks:Running task install... 2024-09-13T16:45:54.792 DEBUG:teuthology.task.install:project ceph 2024-09-13T16:45:54.792 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}} 2024-09-13T16:45:54.793 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'} 2024-09-13T16:45:54.793 INFO:teuthology.task.install:Using flavor: default 2024-09-13T16:45:54.798 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T16:45:54.798 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T16:45:54.799 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-key list | grep Ceph 2024-09-13T16:45:54.800 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-key list | grep Ceph 2024-09-13T16:45:55.023 INFO:teuthology.orchestra.run.smithi003.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T16:45:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T16:45:55.164 INFO:teuthology.orchestra.run.smithi003.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T16:45:55.164 INFO:teuthology.orchestra.run.smithi003.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T16:45:55.166 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T16:45:55.166 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T16:45:55.166 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:45:55.179 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T16:45:55.180 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T16:45:55.180 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T16:45:55.180 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T16:45:55.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:45:55.323 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default/ 2024-09-13T16:45:55.323 INFO:teuthology.task.install.deb:Package version is 18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:45:55.339 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default/ 2024-09-13T16:45:55.339 INFO:teuthology.task.install.deb:Package version is 18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:45:55.445 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:45:55.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T16:45:55.454 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:45:55.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T16:45:55.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get update 2024-09-13T16:45:55.485 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2024-09-13T16:45:55.763 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T16:45:55.763 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T16:45:55.763 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T16:45:55.765 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T16:45:55.771 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T16:45:55.804 INFO:teuthology.orchestra.run.smithi003.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal InRelease 2024-09-13T16:45:55.806 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal InRelease 2024-09-13T16:45:55.826 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T16:45:55.828 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T16:45:55.850 INFO:teuthology.orchestra.run.smithi003.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T16:45:55.852 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T16:45:55.864 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T16:45:55.872 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main i386 Packages [3,435 B] 2024-09-13T16:45:55.874 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T16:45:55.916 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main i386 Packages [3,435 B] 2024-09-13T16:45:55.917 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T16:45:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T16:45:56.029 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T16:45:57.157 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 28.7 kB in 1s (21.6 kB/s) 2024-09-13T16:45:57.161 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 28.7 kB in 1s (21.5 kB/s) 2024-09-13T16:45:58.453 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-09-13T16:45:58.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-850-g50bbe7f3-1focal cephadm=18.2.4-850-g50bbe7f3-1focal ceph-mds=18.2.4-850-g50bbe7f3-1focal ceph-mgr=18.2.4-850-g50bbe7f3-1focal ceph-common=18.2.4-850-g50bbe7f3-1focal ceph-fuse=18.2.4-850-g50bbe7f3-1focal ceph-test=18.2.4-850-g50bbe7f3-1focal ceph-volume=18.2.4-850-g50bbe7f3-1focal radosgw=18.2.4-850-g50bbe7f3-1focal python3-rados=18.2.4-850-g50bbe7f3-1focal python3-rgw=18.2.4-850-g50bbe7f3-1focal python3-cephfs=18.2.4-850-g50bbe7f3-1focal python3-rbd=18.2.4-850-g50bbe7f3-1focal libcephfs2=18.2.4-850-g50bbe7f3-1focal libcephfs-dev=18.2.4-850-g50bbe7f3-1focal librados2=18.2.4-850-g50bbe7f3-1focal librbd1=18.2.4-850-g50bbe7f3-1focal rbd-fuse=18.2.4-850-g50bbe7f3-1focal python3-cephfs=18.2.4-850-g50bbe7f3-1focal cephfs-shell=18.2.4-850-g50bbe7f3-1focal cephfs-top=18.2.4-850-g50bbe7f3-1focal cephfs-mirror=18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:45:58.511 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-13T16:45:58.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-850-g50bbe7f3-1focal cephadm=18.2.4-850-g50bbe7f3-1focal ceph-mds=18.2.4-850-g50bbe7f3-1focal ceph-mgr=18.2.4-850-g50bbe7f3-1focal ceph-common=18.2.4-850-g50bbe7f3-1focal ceph-fuse=18.2.4-850-g50bbe7f3-1focal ceph-test=18.2.4-850-g50bbe7f3-1focal ceph-volume=18.2.4-850-g50bbe7f3-1focal radosgw=18.2.4-850-g50bbe7f3-1focal python3-rados=18.2.4-850-g50bbe7f3-1focal python3-rgw=18.2.4-850-g50bbe7f3-1focal python3-cephfs=18.2.4-850-g50bbe7f3-1focal python3-rbd=18.2.4-850-g50bbe7f3-1focal libcephfs2=18.2.4-850-g50bbe7f3-1focal libcephfs-dev=18.2.4-850-g50bbe7f3-1focal librados2=18.2.4-850-g50bbe7f3-1focal librbd1=18.2.4-850-g50bbe7f3-1focal rbd-fuse=18.2.4-850-g50bbe7f3-1focal python3-cephfs=18.2.4-850-g50bbe7f3-1focal cephfs-shell=18.2.4-850-g50bbe7f3-1focal cephfs-top=18.2.4-850-g50bbe7f3-1focal cephfs-mirror=18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:45:58.617 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-09-13T16:45:58.683 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-13T16:45:58.810 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2024-09-13T16:45:58.811 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2024-09-13T16:45:58.907 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2024-09-13T16:45:58.907 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2024-09-13T16:45:59.008 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:45:59.009 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T16:45:59.009 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:45:59.009 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2024-09-13T16:45:59.009 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T16:45:59.010 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T16:45:59.010 INFO:teuthology.orchestra.run.smithi157.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T16:45:59.010 INFO:teuthology.orchestra.run.smithi157.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T16:45:59.010 INFO:teuthology.orchestra.run.smithi157.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T16:45:59.010 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T16:45:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T16:45:59.012 INFO:teuthology.orchestra.run.smithi157.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T16:45:59.013 INFO:teuthology.orchestra.run.smithi157.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T16:45:59.013 INFO:teuthology.orchestra.run.smithi157.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T16:45:59.013 INFO:teuthology.orchestra.run.smithi157.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T16:45:59.013 INFO:teuthology.orchestra.run.smithi157.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T16:45:59.013 INFO:teuthology.orchestra.run.smithi157.stdout:Recommended packages: 2024-09-13T16:45:59.013 INFO:teuthology.orchestra.run.smithi157.stdout: btrfs-tools 2024-09-13T16:45:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:45:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T16:45:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:45:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2024-09-13T16:45:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T16:45:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T16:45:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T16:45:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T16:45:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T16:45:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2024-09-13T16:45:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout:Recommended packages: 2024-09-13T16:45:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: btrfs-tools 2024-09-13T16:45:59.226 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2024-09-13T16:45:59.227 INFO:teuthology.orchestra.run.smithi157.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T16:45:59.227 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T16:45:59.227 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T16:45:59.227 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-09-13T16:45:59.227 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-09-13T16:45:59.227 INFO:teuthology.orchestra.run.smithi157.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-09-13T16:45:59.227 INFO:teuthology.orchestra.run.smithi157.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-09-13T16:45:59.228 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-09-13T16:45:59.228 INFO:teuthology.orchestra.run.smithi157.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T16:45:59.228 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T16:45:59.228 INFO:teuthology.orchestra.run.smithi157.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T16:45:59.228 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-09-13T16:45:59.228 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T16:45:59.229 INFO:teuthology.orchestra.run.smithi157.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-13T16:45:59.230 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages will be upgraded: 2024-09-13T16:45:59.230 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 librbd1 2024-09-13T16:45:59.316 INFO:teuthology.orchestra.run.smithi157.stdout:2 upgraded, 116 newly installed, 0 to remove and 298 not upgraded. 2024-09-13T16:45:59.316 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 199 MB of archives. 2024-09-13T16:45:59.316 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 979 MB of additional disk space will be used. 2024-09-13T16:45:59.317 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-09-13T16:45:59.335 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T16:45:59.343 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-09-13T16:45:59.344 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2024-09-13T16:45:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T16:45:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T16:45:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T16:45:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-09-13T16:45:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-09-13T16:45:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-09-13T16:45:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-09-13T16:45:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-09-13T16:45:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T16:45:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T16:45:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T16:45:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-09-13T16:45:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T16:45:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-13T16:45:59.347 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages will be upgraded: 2024-09-13T16:45:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 librbd1 2024-09-13T16:45:59.369 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T16:45:59.398 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T16:45:59.401 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T16:45:59.407 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-850-g50bbe7f3-1focal [3,450 kB] 2024-09-13T16:45:59.414 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T16:45:59.436 INFO:teuthology.orchestra.run.smithi003.stdout:2 upgraded, 116 newly installed, 0 to remove and 298 not upgraded. 2024-09-13T16:45:59.436 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 199 MB of archives. 2024-09-13T16:45:59.436 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 979 MB of additional disk space will be used. 2024-09-13T16:45:59.436 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-09-13T16:45:59.456 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T16:45:59.458 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T16:45:59.460 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T16:45:59.463 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-09-13T16:45:59.468 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T16:45:59.471 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T16:45:59.471 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T16:45:59.473 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T16:45:59.473 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T16:45:59.477 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T16:45:59.477 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T16:45:59.479 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T16:45:59.490 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T16:45:59.493 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T16:45:59.494 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T16:45:59.495 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T16:45:59.498 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T16:45:59.499 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T16:45:59.500 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T16:45:59.510 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T16:45:59.511 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T16:45:59.511 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T16:45:59.513 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T16:45:59.519 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T16:45:59.523 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T16:45:59.528 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T16:45:59.530 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-850-g50bbe7f3-1focal [3,450 kB] 2024-09-13T16:45:59.536 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T16:45:59.540 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T16:45:59.540 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T16:45:59.541 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T16:45:59.543 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T16:45:59.544 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T16:45:59.546 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T16:45:59.547 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T16:45:59.549 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T16:45:59.549 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T16:45:59.560 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T16:45:59.563 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T16:45:59.563 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T16:45:59.564 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T16:45:59.577 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T16:45:59.579 INFO:teuthology.orchestra.run.smithi157.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T16:45:59.580 INFO:teuthology.orchestra.run.smithi157.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T16:45:59.581 INFO:teuthology.orchestra.run.smithi157.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T16:45:59.584 INFO:teuthology.orchestra.run.smithi157.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T16:45:59.585 INFO:teuthology.orchestra.run.smithi157.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T16:45:59.585 INFO:teuthology.orchestra.run.smithi157.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-850-g50bbe7f3-1focal [4,041 kB] 2024-09-13T16:45:59.585 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T16:45:59.588 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T16:45:59.594 INFO:teuthology.orchestra.run.smithi157.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T16:45:59.595 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T16:45:59.596 INFO:teuthology.orchestra.run.smithi157.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T16:45:59.597 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T16:45:59.598 INFO:teuthology.orchestra.run.smithi157.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T16:45:59.598 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T16:45:59.598 INFO:teuthology.orchestra.run.smithi157.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T16:45:59.613 INFO:teuthology.orchestra.run.smithi157.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T16:45:59.627 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T16:45:59.628 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T16:45:59.634 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T16:45:59.634 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T16:45:59.635 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T16:45:59.643 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T16:45:59.643 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T16:45:59.644 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T16:45:59.645 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T16:45:59.646 INFO:teuthology.orchestra.run.smithi157.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-850-g50bbe7f3-1focal [747 kB] 2024-09-13T16:45:59.660 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T16:45:59.660 INFO:teuthology.orchestra.run.smithi003.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T16:45:59.661 INFO:teuthology.orchestra.run.smithi157.stdout:Get:56 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-850-g50bbe7f3-1focal [386 kB] 2024-09-13T16:45:59.662 INFO:teuthology.orchestra.run.smithi003.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T16:45:59.662 INFO:teuthology.orchestra.run.smithi003.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T16:45:59.663 INFO:teuthology.orchestra.run.smithi003.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T16:45:59.664 INFO:teuthology.orchestra.run.smithi003.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T16:45:59.668 INFO:teuthology.orchestra.run.smithi157.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-850-g50bbe7f3-1focal [31.9 kB] 2024-09-13T16:45:59.668 INFO:teuthology.orchestra.run.smithi157.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-850-g50bbe7f3-1focal [201 kB] 2024-09-13T16:45:59.671 INFO:teuthology.orchestra.run.smithi157.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-850-g50bbe7f3-1focal [62.5 kB] 2024-09-13T16:45:59.672 INFO:teuthology.orchestra.run.smithi157.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-850-g50bbe7f3-1focal [367 kB] 2024-09-13T16:45:59.677 INFO:teuthology.orchestra.run.smithi003.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T16:45:59.679 INFO:teuthology.orchestra.run.smithi157.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-850-g50bbe7f3-1focal [6,627 kB] 2024-09-13T16:45:59.681 INFO:teuthology.orchestra.run.smithi003.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T16:45:59.682 INFO:teuthology.orchestra.run.smithi003.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T16:45:59.683 INFO:teuthology.orchestra.run.smithi003.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T16:45:59.696 INFO:teuthology.orchestra.run.smithi003.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T16:45:59.696 INFO:teuthology.orchestra.run.smithi003.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T16:45:59.698 INFO:teuthology.orchestra.run.smithi003.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T16:45:59.699 INFO:teuthology.orchestra.run.smithi003.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T16:45:59.702 INFO:teuthology.orchestra.run.smithi003.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T16:45:59.702 INFO:teuthology.orchestra.run.smithi003.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T16:45:59.712 INFO:teuthology.orchestra.run.smithi003.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T16:45:59.714 INFO:teuthology.orchestra.run.smithi003.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T16:45:59.715 INFO:teuthology.orchestra.run.smithi003.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T16:45:59.717 INFO:teuthology.orchestra.run.smithi003.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T16:45:59.728 INFO:teuthology.orchestra.run.smithi157.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T16:45:59.729 INFO:teuthology.orchestra.run.smithi003.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T16:45:59.731 INFO:teuthology.orchestra.run.smithi003.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T16:45:59.731 INFO:teuthology.orchestra.run.smithi003.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T16:45:59.732 INFO:teuthology.orchestra.run.smithi003.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T16:45:59.735 INFO:teuthology.orchestra.run.smithi003.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T16:45:59.736 INFO:teuthology.orchestra.run.smithi003.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T16:45:59.747 INFO:teuthology.orchestra.run.smithi003.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T16:45:59.750 INFO:teuthology.orchestra.run.smithi003.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T16:45:59.752 INFO:teuthology.orchestra.run.smithi003.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T16:45:59.753 INFO:teuthology.orchestra.run.smithi003.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T16:45:59.755 INFO:teuthology.orchestra.run.smithi157.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T16:45:59.758 INFO:teuthology.orchestra.run.smithi157.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T16:45:59.765 INFO:teuthology.orchestra.run.smithi003.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T16:45:59.777 INFO:teuthology.orchestra.run.smithi157.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T16:45:59.777 INFO:teuthology.orchestra.run.smithi157.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T16:45:59.777 INFO:teuthology.orchestra.run.smithi157.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T16:45:59.778 INFO:teuthology.orchestra.run.smithi157.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T16:45:59.779 INFO:teuthology.orchestra.run.smithi157.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T16:45:59.779 INFO:teuthology.orchestra.run.smithi157.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T16:45:59.780 INFO:teuthology.orchestra.run.smithi157.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T16:45:59.781 INFO:teuthology.orchestra.run.smithi157.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T16:45:59.794 INFO:teuthology.orchestra.run.smithi157.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T16:45:59.799 INFO:teuthology.orchestra.run.smithi157.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T16:45:59.804 INFO:teuthology.orchestra.run.smithi157.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T16:45:59.807 INFO:teuthology.orchestra.run.smithi157.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T16:45:59.809 INFO:teuthology.orchestra.run.smithi157.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-850-g50bbe7f3-1focal [113 kB] 2024-09-13T16:45:59.810 INFO:teuthology.orchestra.run.smithi157.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T16:45:59.811 INFO:teuthology.orchestra.run.smithi157.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-850-g50bbe7f3-1focal [479 kB] 2024-09-13T16:45:59.812 INFO:teuthology.orchestra.run.smithi157.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T16:45:59.818 INFO:teuthology.orchestra.run.smithi157.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T16:45:59.819 INFO:teuthology.orchestra.run.smithi157.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-850-g50bbe7f3-1focal [25.2 MB] 2024-09-13T16:45:59.824 INFO:teuthology.orchestra.run.smithi157.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-09-13T16:45:59.825 INFO:teuthology.orchestra.run.smithi157.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T16:45:59.854 INFO:teuthology.orchestra.run.smithi157.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T16:45:59.859 INFO:teuthology.orchestra.run.smithi157.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T16:45:59.862 INFO:teuthology.orchestra.run.smithi157.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T16:45:59.873 INFO:teuthology.orchestra.run.smithi003.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T16:45:59.893 INFO:teuthology.orchestra.run.smithi003.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T16:45:59.895 INFO:teuthology.orchestra.run.smithi003.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T16:45:59.909 INFO:teuthology.orchestra.run.smithi003.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T16:45:59.909 INFO:teuthology.orchestra.run.smithi003.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T16:45:59.910 INFO:teuthology.orchestra.run.smithi003.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T16:45:59.910 INFO:teuthology.orchestra.run.smithi003.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T16:45:59.911 INFO:teuthology.orchestra.run.smithi003.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T16:45:59.911 INFO:teuthology.orchestra.run.smithi003.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T16:45:59.911 INFO:teuthology.orchestra.run.smithi003.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T16:45:59.912 INFO:teuthology.orchestra.run.smithi003.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T16:45:59.928 INFO:teuthology.orchestra.run.smithi157.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T16:45:59.929 INFO:teuthology.orchestra.run.smithi003.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T16:45:59.929 INFO:teuthology.orchestra.run.smithi003.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T16:45:59.930 INFO:teuthology.orchestra.run.smithi157.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T16:45:59.930 INFO:teuthology.orchestra.run.smithi157.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T16:45:59.931 INFO:teuthology.orchestra.run.smithi157.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T16:45:59.934 INFO:teuthology.orchestra.run.smithi003.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T16:45:59.945 INFO:teuthology.orchestra.run.smithi003.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T16:45:59.951 INFO:teuthology.orchestra.run.smithi003.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T16:45:59.952 INFO:teuthology.orchestra.run.smithi157.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T16:45:59.953 INFO:teuthology.orchestra.run.smithi157.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T16:45:59.955 INFO:teuthology.orchestra.run.smithi003.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T16:45:59.955 INFO:teuthology.orchestra.run.smithi157.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T16:45:59.955 INFO:teuthology.orchestra.run.smithi157.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T16:45:59.956 INFO:teuthology.orchestra.run.smithi157.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T16:45:59.958 INFO:teuthology.orchestra.run.smithi003.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T16:45:59.961 INFO:teuthology.orchestra.run.smithi003.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-09-13T16:45:59.961 INFO:teuthology.orchestra.run.smithi003.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T16:45:59.980 INFO:teuthology.orchestra.run.smithi003.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T16:45:59.983 INFO:teuthology.orchestra.run.smithi003.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T16:45:59.986 INFO:teuthology.orchestra.run.smithi003.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T16:46:00.034 INFO:teuthology.orchestra.run.smithi003.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T16:46:00.035 INFO:teuthology.orchestra.run.smithi003.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T16:46:00.036 INFO:teuthology.orchestra.run.smithi003.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T16:46:00.037 INFO:teuthology.orchestra.run.smithi003.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T16:46:00.051 INFO:teuthology.orchestra.run.smithi003.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T16:46:00.051 INFO:teuthology.orchestra.run.smithi003.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T16:46:00.052 INFO:teuthology.orchestra.run.smithi003.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T16:46:00.053 INFO:teuthology.orchestra.run.smithi003.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T16:46:00.053 INFO:teuthology.orchestra.run.smithi003.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T16:46:00.124 INFO:teuthology.orchestra.run.smithi003.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-850-g50bbe7f3-1focal [4,041 kB] 2024-09-13T16:46:00.288 INFO:teuthology.orchestra.run.smithi157.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-850-g50bbe7f3-1focal [9,614 kB] 2024-09-13T16:46:00.475 INFO:teuthology.orchestra.run.smithi157.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-850-g50bbe7f3-1focal [238 kB] 2024-09-13T16:46:00.479 INFO:teuthology.orchestra.run.smithi157.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-850-g50bbe7f3-1focal [148 kB] 2024-09-13T16:46:00.480 INFO:teuthology.orchestra.run.smithi157.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-850-g50bbe7f3-1focal [1,575 kB] 2024-09-13T16:46:00.506 INFO:teuthology.orchestra.run.smithi157.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-850-g50bbe7f3-1focal [6,639 kB] 2024-09-13T16:46:00.518 INFO:teuthology.orchestra.run.smithi003.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-850-g50bbe7f3-1focal [747 kB] 2024-09-13T16:46:00.614 INFO:teuthology.orchestra.run.smithi003.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-850-g50bbe7f3-1focal [386 kB] 2024-09-13T16:46:00.627 INFO:teuthology.orchestra.run.smithi003.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-850-g50bbe7f3-1focal [31.9 kB] 2024-09-13T16:46:00.628 INFO:teuthology.orchestra.run.smithi003.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-850-g50bbe7f3-1focal [201 kB] 2024-09-13T16:46:00.646 INFO:teuthology.orchestra.run.smithi157.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-850-g50bbe7f3-1focal [24.5 MB] 2024-09-13T16:46:00.670 INFO:teuthology.orchestra.run.smithi003.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-850-g50bbe7f3-1focal [62.5 kB] 2024-09-13T16:46:00.673 INFO:teuthology.orchestra.run.smithi003.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-850-g50bbe7f3-1focal [367 kB] 2024-09-13T16:46:00.683 INFO:teuthology.orchestra.run.smithi003.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-850-g50bbe7f3-1focal [6,627 kB] 2024-09-13T16:46:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-850-g50bbe7f3-1focal [113 kB] 2024-09-13T16:46:01.170 INFO:teuthology.orchestra.run.smithi157.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-850-g50bbe7f3-1focal [13.7 kB] 2024-09-13T16:46:01.170 INFO:teuthology.orchestra.run.smithi157.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-850-g50bbe7f3-1focal [856 kB] 2024-09-13T16:46:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-850-g50bbe7f3-1focal [479 kB] 2024-09-13T16:46:01.188 INFO:teuthology.orchestra.run.smithi157.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-850-g50bbe7f3-1focal [2,296 kB] 2024-09-13T16:46:01.211 INFO:teuthology.orchestra.run.smithi003.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-850-g50bbe7f3-1focal [25.2 MB] 2024-09-13T16:46:01.245 INFO:teuthology.orchestra.run.smithi157.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-850-g50bbe7f3-1focal [239 kB] 2024-09-13T16:46:01.250 INFO:teuthology.orchestra.run.smithi157.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-850-g50bbe7f3-1focal [143 kB] 2024-09-13T16:46:01.254 INFO:teuthology.orchestra.run.smithi157.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-850-g50bbe7f3-1focal [7,015 kB] 2024-09-13T16:46:01.403 INFO:teuthology.orchestra.run.smithi157.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-850-g50bbe7f3-1focal [7,873 kB] 2024-09-13T16:46:01.597 INFO:teuthology.orchestra.run.smithi157.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-850-g50bbe7f3-1focal [13.9 kB] 2024-09-13T16:46:01.598 INFO:teuthology.orchestra.run.smithi157.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-850-g50bbe7f3-1focal [46.9 MB] 2024-09-13T16:46:02.367 INFO:teuthology.orchestra.run.smithi003.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-850-g50bbe7f3-1focal [9,614 kB] 2024-09-13T16:46:02.664 INFO:teuthology.orchestra.run.smithi003.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-850-g50bbe7f3-1focal [238 kB] 2024-09-13T16:46:02.669 INFO:teuthology.orchestra.run.smithi003.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-850-g50bbe7f3-1focal [148 kB] 2024-09-13T16:46:02.671 INFO:teuthology.orchestra.run.smithi003.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-850-g50bbe7f3-1focal [1,575 kB] 2024-09-13T16:46:02.710 INFO:teuthology.orchestra.run.smithi003.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-850-g50bbe7f3-1focal [6,639 kB] 2024-09-13T16:46:02.947 INFO:teuthology.orchestra.run.smithi003.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-850-g50bbe7f3-1focal [24.5 MB] 2024-09-13T16:46:02.996 INFO:teuthology.orchestra.run.smithi157.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-850-g50bbe7f3-1focal [120 kB] 2024-09-13T16:46:02.997 INFO:teuthology.orchestra.run.smithi157.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.4-850-g50bbe7f3-1focal [270 kB] 2024-09-13T16:46:03.000 INFO:teuthology.orchestra.run.smithi157.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.4-850-g50bbe7f3-1focal [33.0 kB] 2024-09-13T16:46:03.000 INFO:teuthology.orchestra.run.smithi157.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.4-850-g50bbe7f3-1focal [22.8 kB] 2024-09-13T16:46:03.002 INFO:teuthology.orchestra.run.smithi157.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-850-g50bbe7f3-1focal [37.4 kB] 2024-09-13T16:46:03.003 INFO:teuthology.orchestra.run.smithi157.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-850-g50bbe7f3-1focal [13.5 MB] 2024-09-13T16:46:03.452 INFO:teuthology.orchestra.run.smithi157.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-850-g50bbe7f3-1focal [107 kB] 2024-09-13T16:46:03.599 INFO:teuthology.orchestra.run.smithi003.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-850-g50bbe7f3-1focal [13.7 kB] 2024-09-13T16:46:03.599 INFO:teuthology.orchestra.run.smithi003.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-850-g50bbe7f3-1focal [856 kB] 2024-09-13T16:46:03.608 INFO:teuthology.orchestra.run.smithi003.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-850-g50bbe7f3-1focal [2,296 kB] 2024-09-13T16:46:03.642 INFO:teuthology.orchestra.run.smithi003.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-850-g50bbe7f3-1focal [239 kB] 2024-09-13T16:46:03.645 INFO:teuthology.orchestra.run.smithi003.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-850-g50bbe7f3-1focal [143 kB] 2024-09-13T16:46:03.647 INFO:teuthology.orchestra.run.smithi003.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-850-g50bbe7f3-1focal [7,015 kB] 2024-09-13T16:46:03.765 INFO:teuthology.orchestra.run.smithi003.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-850-g50bbe7f3-1focal [7,873 kB] 2024-09-13T16:46:03.895 INFO:teuthology.orchestra.run.smithi003.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-850-g50bbe7f3-1focal [13.9 kB] 2024-09-13T16:46:03.896 INFO:teuthology.orchestra.run.smithi003.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-850-g50bbe7f3-1focal [46.9 MB] 2024-09-13T16:46:04.383 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 199 MB in 4s (47.3 MB/s) 2024-09-13T16:46:04.561 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T16:46:04.632 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107754 files and directories currently installed.) 2024-09-13T16:46:04.634 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-09-13T16:46:04.678 INFO:teuthology.orchestra.run.smithi003.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-850-g50bbe7f3-1focal [120 kB] 2024-09-13T16:46:04.680 INFO:teuthology.orchestra.run.smithi003.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.4-850-g50bbe7f3-1focal [270 kB] 2024-09-13T16:46:04.682 INFO:teuthology.orchestra.run.smithi003.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.4-850-g50bbe7f3-1focal [33.0 kB] 2024-09-13T16:46:04.682 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-09-13T16:46:04.683 INFO:teuthology.orchestra.run.smithi003.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.4-850-g50bbe7f3-1focal [22.8 kB] 2024-09-13T16:46:04.683 INFO:teuthology.orchestra.run.smithi003.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-850-g50bbe7f3-1focal [37.4 kB] 2024-09-13T16:46:04.683 INFO:teuthology.orchestra.run.smithi003.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-850-g50bbe7f3-1focal [13.5 MB] 2024-09-13T16:46:04.895 INFO:teuthology.orchestra.run.smithi003.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-850-g50bbe7f3-1focal [107 kB] 2024-09-13T16:46:05.051 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T16:46:05.071 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T16:46:05.109 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T16:46:05.454 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T16:46:05.472 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-09-13T16:46:05.512 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-09-13T16:46:05.850 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 199 MB in 6s (36.0 MB/s) 2024-09-13T16:46:05.907 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T16:46:05.924 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T16:46:06.008 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T16:46:06.032 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T16:46:06.078 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107754 files and directories currently installed.) 2024-09-13T16:46:06.081 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-09-13T16:46:06.121 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-09-13T16:46:06.444 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T16:46:06.465 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T16:46:06.466 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T16:46:06.484 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T16:46:06.510 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T16:46:06.556 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T16:46:06.876 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T16:46:06.895 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-09-13T16:46:06.905 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T16:46:06.923 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T16:46:06.959 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-09-13T16:46:06.963 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T16:46:07.279 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T16:46:07.296 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T16:46:07.397 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T16:46:07.483 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T16:46:07.501 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T16:46:07.579 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:46:07.807 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T16:46:07.830 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T16:46:07.874 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T16:46:08.119 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T16:46:08.138 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T16:46:08.177 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:46:08.277 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T16:46:08.297 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T16:46:08.335 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T16:46:08.589 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T16:46:08.607 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T16:46:08.647 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:46:08.872 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T16:46:08.897 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T16:46:08.955 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:46:09.109 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T16:46:09.127 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T16:46:09.167 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T16:46:09.491 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T16:46:09.504 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T16:46:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:46:09.547 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../010-librbd1_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:09.712 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librbd1 (18.2.4-850-g50bbe7f3-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T16:46:09.945 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T16:46:09.964 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T16:46:10.028 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:46:10.481 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T16:46:10.486 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../011-librados2_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:10.500 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T16:46:10.538 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T16:46:10.607 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librados2 (18.2.4-850-g50bbe7f3-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T16:46:10.936 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../010-librbd1_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:11.059 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librbd1 (18.2.4-850-g50bbe7f3-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T16:46:11.420 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T16:46:11.438 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../012-libcephfs2_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:11.477 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:11.741 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../011-librados2_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:11.846 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librados2 (18.2.4-850-g50bbe7f3-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T16:46:12.055 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rados. 2024-09-13T16:46:12.075 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../013-python3-rados_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rados (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:12.542 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T16:46:12.560 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:12.599 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-argparse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:12.625 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T16:46:12.647 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../012-libcephfs2_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:12.724 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libcephfs2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:12.911 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T16:46:12.928 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../015-python3-cephfs_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:12.969 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cephfs (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:13.130 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rados. 2024-09-13T16:46:13.148 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../013-python3-rados_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:13.185 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rados (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:13.330 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T16:46:13.348 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:13.388 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-common (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:13.580 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T16:46:13.600 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:13.638 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-ceph-argparse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:13.766 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T16:46:13.785 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T16:46:13.824 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T16:46:14.102 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T16:46:14.121 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../018-python3-rbd_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:14.160 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rbd (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:14.258 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T16:46:14.278 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../015-python3-cephfs_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:14.332 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cephfs (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:14.554 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T16:46:14.572 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T16:46:14.612 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T16:46:14.702 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T16:46:14.720 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:14.760 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-ceph-common (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:15.057 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T16:46:15.079 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T16:46:15.123 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T16:46:15.146 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T16:46:15.165 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T16:46:15.204 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T16:46:15.501 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T16:46:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T16:46:15.520 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T16:46:15.525 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../018-python3-rbd_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:15.559 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T16:46:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rbd (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:15.929 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librgw2. 2024-09-13T16:46:15.948 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../022-librgw2_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:15.976 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T16:46:15.978 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librgw2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:15.995 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T16:46:16.034 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T16:46:16.446 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T16:46:16.464 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T16:46:16.504 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T16:46:16.889 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T16:46:16.898 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T16:46:16.907 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../023-python3-rgw_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:16.917 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T16:46:16.947 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rgw (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:16.996 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T16:46:17.342 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liboath0. 2024-09-13T16:46:17.360 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T16:46:17.400 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T16:46:17.409 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librgw2. 2024-09-13T16:46:17.429 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../022-librgw2_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:17.467 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librgw2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:17.770 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T16:46:17.789 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../025-libradosstriper1_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:17.853 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libradosstriper1 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:18.348 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-common. 2024-09-13T16:46:18.368 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../026-ceph-common_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:18.406 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-common (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:18.428 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T16:46:18.442 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../023-python3-rgw_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:18.478 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rgw (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:18.881 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liboath0. 2024-09-13T16:46:18.900 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T16:46:18.964 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T16:46:19.333 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T16:46:19.352 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../025-libradosstriper1_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:19.391 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libradosstriper1 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:19.900 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-common. 2024-09-13T16:46:19.920 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../026-ceph-common_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:19.958 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-common (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:20.953 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-base. 2024-09-13T16:46:20.972 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../027-ceph-base_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:21.032 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-base (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:22.404 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T16:46:22.422 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T16:46:22.462 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T16:46:22.613 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-base. 2024-09-13T16:46:22.632 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../027-ceph-base_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:22.693 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-base (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:22.848 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T16:46:22.868 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T16:46:22.915 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T16:46:23.301 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T16:46:23.321 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T16:46:23.359 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T16:46:23.729 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T16:46:23.748 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T16:46:23.829 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T16:46:24.026 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T16:46:24.045 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T16:46:24.075 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T16:46:24.156 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-mako. 2024-09-13T16:46:24.169 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T16:46:24.248 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T16:46:24.481 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T16:46:24.500 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T16:46:24.539 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T16:46:24.652 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T16:46:24.671 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T16:46:24.710 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T16:46:24.918 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T16:46:24.937 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T16:46:24.975 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T16:46:25.071 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T16:46:25.088 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T16:46:25.129 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T16:46:25.337 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T16:46:25.357 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T16:46:25.395 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T16:46:25.501 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webob. 2024-09-13T16:46:25.519 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T16:46:25.559 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T16:46:25.752 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-mako. 2024-09-13T16:46:25.772 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T16:46:25.810 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T16:46:25.947 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T16:46:25.965 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T16:46:26.005 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T16:46:26.205 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T16:46:26.224 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T16:46:26.263 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T16:46:26.367 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T16:46:26.385 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T16:46:26.424 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T16:46:26.608 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T16:46:26.625 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T16:46:26.689 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T16:46:26.818 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T16:46:26.837 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T16:46:26.880 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T16:46:27.017 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webob. 2024-09-13T16:46:27.035 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T16:46:27.100 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T16:46:27.470 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T16:46:27.491 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T16:46:27.536 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T16:46:27.604 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T16:46:27.622 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T16:46:27.662 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T16:46:27.897 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T16:46:27.917 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T16:46:27.955 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T16:46:28.032 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-paste. 2024-09-13T16:46:28.050 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T16:46:28.090 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T16:46:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T16:46:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T16:46:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T16:46:28.477 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T16:46:28.495 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T16:46:28.535 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T16:46:28.796 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T16:46:28.813 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T16:46:28.853 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T16:46:28.888 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T16:46:28.908 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T16:46:28.946 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T16:46:29.223 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-paste. 2024-09-13T16:46:29.241 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T16:46:29.281 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T16:46:29.301 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T16:46:29.321 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T16:46:29.359 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T16:46:29.737 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T16:46:29.756 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T16:46:29.795 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T16:46:30.183 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T16:46:30.201 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T16:46:30.258 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T16:46:30.418 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T16:46:30.437 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T16:46:30.475 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T16:46:30.561 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T16:46:30.580 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:30.635 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-modules-core (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:30.829 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T16:46:30.849 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T16:46:30.887 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T16:46:31.099 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T16:46:31.116 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:31.157 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T16:46:31.275 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T16:46:31.339 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T16:46:31.543 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T16:46:31.561 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T16:46:31.601 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T16:46:31.726 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T16:46:31.745 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T16:46:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T16:46:31.954 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T16:46:31.975 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T16:46:32.020 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T16:46:32.189 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T16:46:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T16:46:32.246 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T16:46:32.575 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T16:46:32.595 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:32.632 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-modules-core (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:33.104 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T16:46:33.124 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:33.162 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:33.190 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T16:46:33.203 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T16:46:33.248 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T16:46:33.540 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T16:46:33.553 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T16:46:33.635 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T16:46:33.636 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T16:46:33.653 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../051-ceph-mgr_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:33.693 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:34.015 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T16:46:34.033 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T16:46:34.085 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T16:46:34.206 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T16:46:34.224 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../052-ceph-mon_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:34.263 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mon (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:34.506 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T16:46:34.525 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T16:46:34.563 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T16:46:34.958 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T16:46:34.977 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../051-ceph-mgr_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:35.016 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:35.292 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T16:46:35.309 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../053-ceph-osd_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:35.350 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-osd (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:35.561 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T16:46:35.581 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../052-ceph-mon_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:35.619 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mon (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:36.623 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T16:46:36.642 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../053-ceph-osd_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:36.714 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-osd (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:37.562 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph. 2024-09-13T16:46:37.580 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../054-ceph_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:37.619 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:38.014 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T16:46:38.032 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../055-ceph-fuse_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:38.072 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-fuse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:38.542 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T16:46:38.561 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../056-ceph-mds_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:38.599 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mds (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:38.825 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph. 2024-09-13T16:46:38.846 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../054-ceph_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:38.883 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:39.161 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephadm. 2024-09-13T16:46:39.180 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../057-cephadm_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:39.219 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephadm (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:39.286 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T16:46:39.305 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../055-ceph-fuse_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:39.344 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-fuse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:39.580 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T16:46:39.599 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T16:46:39.638 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T16:46:39.789 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T16:46:39.808 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../056-ceph-mds_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:39.863 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mds (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:40.058 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T16:46:40.079 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T16:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T16:46:40.408 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephadm. 2024-09-13T16:46:40.429 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../057-cephadm_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:46:40.466 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephadm (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:40.527 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T16:46:40.545 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:40.576 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-cephadm (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:40.828 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T16:46:40.844 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T16:46:40.885 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T16:46:40.965 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T16:46:40.981 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T16:46:41.038 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T16:46:41.305 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T16:46:41.318 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T16:46:41.349 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-attr. 2024-09-13T16:46:41.371 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-09-13T16:46:41.388 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T16:46:41.424 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T16:46:41.783 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T16:46:41.800 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:41.819 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T16:46:41.839 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T16:46:41.840 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-cephadm (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:41.876 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T16:46:42.230 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-py. 2024-09-13T16:46:42.244 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T16:46:42.248 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T16:46:42.261 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T16:46:42.287 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T16:46:42.302 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T16:46:42.672 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-attr. 2024-09-13T16:46:42.691 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T16:46:42.692 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-09-13T16:46:42.710 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T16:46:42.730 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T16:46:42.748 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T16:46:43.085 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T16:46:43.104 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-09-13T16:46:43.133 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T16:46:43.143 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T16:46:43.155 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T16:46:43.200 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T16:46:43.488 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T16:46:43.508 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T16:46:43.546 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T16:46:43.578 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-py. 2024-09-13T16:46:43.596 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T16:46:43.635 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T16:46:43.942 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T16:46:43.961 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T16:46:43.999 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T16:46:44.014 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T16:46:44.033 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T16:46:44.089 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T16:46:44.302 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T16:46:44.322 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T16:46:44.360 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T16:46:44.450 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T16:46:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-09-13T16:46:44.508 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T16:46:44.679 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T16:46:44.700 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T16:46:44.746 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T16:46:44.861 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T16:46:44.878 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T16:46:44.919 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T16:46:45.132 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-routes. 2024-09-13T16:46:45.152 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-09-13T16:46:45.190 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T16:46:45.577 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T16:46:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T16:46:45.596 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T16:46:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-dashboard (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:45.647 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T16:46:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T16:46:45.931 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T16:46:45.974 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T16:46:46.314 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T16:46:46.332 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T16:46:46.372 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T16:46:46.741 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-routes. 2024-09-13T16:46:46.760 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-09-13T16:46:46.799 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T16:46:47.100 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T16:46:47.114 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T16:46:47.153 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T16:46:47.159 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T16:46:47.172 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:47.235 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-dashboard (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:47.537 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T16:46:47.557 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T16:46:47.595 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T16:46:47.965 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T16:46:47.984 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T16:46:48.022 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T16:46:48.641 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T16:46:48.660 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T16:46:48.698 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T16:46:49.219 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T16:46:49.237 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T16:46:49.277 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T16:46:49.482 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T16:46:49.501 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T16:46:49.540 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T16:46:49.622 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T16:46:49.641 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T16:46:49.679 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T16:46:50.177 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T16:46:50.195 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T16:46:50.243 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T16:46:50.625 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T16:46:50.643 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T16:46:50.708 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T16:46:51.192 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T16:46:51.211 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T16:46:51.249 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T16:46:51.510 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T16:46:51.530 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:51.568 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:51.861 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T16:46:51.881 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T16:46:51.946 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T16:46:52.367 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T16:46:52.384 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T16:46:52.424 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T16:46:52.829 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T16:46:52.843 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T16:46:52.887 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T16:46:53.208 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T16:46:53.226 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:53.265 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:53.334 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T16:46:53.352 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T16:46:53.430 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T16:46:53.830 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T16:46:53.849 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T16:46:53.887 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T16:46:54.267 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T16:46:54.285 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-09-13T16:46:54.324 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T16:46:54.501 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T16:46:54.517 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T16:46:54.559 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T16:46:54.694 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T16:46:54.707 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T16:46:54.752 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T16:46:55.057 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T16:46:55.077 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T16:46:55.131 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T16:46:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T16:46:55.152 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T16:46:55.197 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T16:46:55.494 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T16:46:55.514 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T16:46:55.551 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T16:46:55.566 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T16:46:55.585 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T16:46:55.625 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T16:46:55.947 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T16:46:55.966 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-09-13T16:46:56.005 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T16:46:56.014 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T16:46:56.034 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T16:46:56.072 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T16:46:56.394 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T16:46:56.413 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T16:46:56.451 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T16:46:56.452 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T16:46:56.472 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T16:46:56.517 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T16:46:56.856 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T16:46:56.877 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T16:46:56.896 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T16:46:56.913 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T16:46:56.913 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T16:46:56.986 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T16:46:57.267 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T16:46:57.288 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T16:46:57.367 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T16:46:57.620 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T16:46:57.640 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:57.677 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:57.773 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T16:46:57.787 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T16:46:57.831 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T16:46:58.218 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T16:46:58.240 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T16:46:58.275 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T16:46:58.339 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T16:46:58.357 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T16:46:58.396 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T16:46:58.645 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T16:46:58.665 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T16:46:58.703 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T16:46:58.792 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T16:46:58.810 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:46:58.860 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T16:46:59.180 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package jq. 2024-09-13T16:46:59.198 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:46:59.237 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T16:46:59.402 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T16:46:59.415 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:46:59.460 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:46:59.540 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package socat. 2024-09-13T16:46:59.560 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T16:46:59.598 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T16:46:59.855 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T16:46:59.873 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T16:46:59.912 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T16:46:59.952 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T16:46:59.972 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T16:47:00.009 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T16:47:00.318 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T16:47:00.338 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:47:00.375 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T16:47:00.464 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-test. 2024-09-13T16:47:00.482 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../096-ceph-test_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:00.546 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-test (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:00.704 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package jq. 2024-09-13T16:47:00.726 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:47:00.770 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T16:47:01.073 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package socat. 2024-09-13T16:47:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T16:47:01.139 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T16:47:01.510 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T16:47:01.532 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T16:47:01.609 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T16:47:02.041 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-test. 2024-09-13T16:47:02.057 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../096-ceph-test_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:02.096 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-test (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:04.501 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T16:47:04.519 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:47:04.558 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-volume (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:04.988 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T16:47:05.007 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:05.045 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-mirror (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:05.441 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T16:47:05.460 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-09-13T16:47:05.498 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-09-13T16:47:05.810 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T16:47:05.829 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../100-cephfs-shell_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:47:05.867 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-shell (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:06.093 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T16:47:06.113 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:47:06.150 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-volume (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:06.162 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T16:47:06.182 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../101-cephfs-top_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:47:06.220 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-top (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:06.531 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T16:47:06.546 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T16:47:06.589 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T16:47:06.589 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T16:47:06.606 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:06.646 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-mirror (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:07.050 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T16:47:07.069 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-09-13T16:47:07.107 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-09-13T16:47:07.126 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T16:47:07.148 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:07.192 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs-dev (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:07.477 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T16:47:07.499 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../100-cephfs-shell_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:47:07.543 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-shell (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:07.629 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T16:47:07.649 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T16:47:07.761 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T16:47:07.846 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T16:47:07.866 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../101-cephfs-top_18.2.4-850-g50bbe7f3-1focal_all.deb ... 2024-09-13T16:47:07.904 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-top (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:08.141 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T16:47:08.159 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T16:47:08.199 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T16:47:08.199 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T16:47:08.221 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T16:47:08.273 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T16:47:08.653 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T16:47:08.674 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T16:47:08.719 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T16:47:08.852 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T16:47:08.874 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:08.918 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libcephfs-dev (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:09.388 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T16:47:09.407 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T16:47:09.480 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T16:47:09.851 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T16:47:09.874 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T16:47:09.942 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T16:47:10.371 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T16:47:10.392 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T16:47:10.437 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T16:47:10.789 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T16:47:10.810 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T16:47:10.863 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T16:47:11.260 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T16:47:11.278 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T16:47:11.316 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T16:47:11.678 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T16:47:11.697 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T16:47:11.735 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T16:47:12.156 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T16:47:12.174 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T16:47:12.213 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T16:47:12.513 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T16:47:12.531 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T16:47:12.608 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T16:47:12.750 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T16:47:12.770 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T16:47:12.808 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T16:47:13.015 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T16:47:13.033 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T16:47:13.073 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T16:47:13.162 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T16:47:13.185 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T16:47:13.253 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T16:47:13.427 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T16:47:13.447 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T16:47:13.484 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T16:47:13.665 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T16:47:13.686 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T16:47:13.731 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T16:47:13.871 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T16:47:13.887 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T16:47:13.929 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T16:47:14.075 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T16:47:14.093 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T16:47:14.133 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T16:47:14.424 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T16:47:14.443 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T16:47:14.469 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T16:47:14.482 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T16:47:14.489 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T16:47:14.528 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T16:47:14.836 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T16:47:14.857 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T16:47:14.944 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T16:47:15.298 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package radosgw. 2024-09-13T16:47:15.314 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../116-radosgw_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:15.339 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T16:47:15.356 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking radosgw (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:15.358 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T16:47:15.396 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T16:47:15.741 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T16:47:15.761 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T16:47:16.165 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T16:47:16.477 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T16:47:16.491 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T16:47:16.559 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T16:47:16.759 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T16:47:16.778 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../117-rbd-fuse_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:16.817 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking rbd-fuse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:17.189 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T16:47:17.272 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package radosgw. 2024-09-13T16:47:17.294 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../116-radosgw_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:17.346 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking radosgw (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T16:47:17.723 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T16:47:17.849 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T16:47:18.092 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephadm (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T16:47:18.771 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../117-rbd-fuse_18.2.4-850-g50bbe7f3-1focal_amd64.deb ... 2024-09-13T16:47:18.816 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking rbd-fuse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:18.875 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user cephadm....done 2024-09-13T16:47:19.065 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T16:47:19.191 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T16:47:19.204 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T16:47:19.660 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T16:47:19.662 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T16:47:19.789 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T16:47:19.914 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T16:47:20.044 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T16:47:20.166 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephadm (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:20.170 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T16:47:20.297 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T16:47:20.690 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T16:47:20.845 INFO:teuthology.orchestra.run.smithi003.stdout:Adding system user cephadm....done 2024-09-13T16:47:21.039 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T16:47:21.190 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T16:47:21.217 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T16:47:21.568 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T16:47:21.625 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T16:47:21.953 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-09-13T16:47:21.993 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T16:47:22.119 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T16:47:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T16:47:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T16:47:22.672 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T16:47:22.680 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T16:47:23.031 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-argparse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:23.207 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T16:47:23.391 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T16:47:23.517 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-09-13T16:47:23.567 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T16:47:23.884 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T16:47:23.918 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-09-13T16:47:24.260 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T16:47:24.294 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T16:47:24.587 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T16:47:24.670 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T16:47:24.713 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T16:47:25.071 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-ceph-argparse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:25.222 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T16:47:25.456 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T16:47:25.581 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-09-13T16:47:25.582 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T16:47:25.941 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T16:47:25.949 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T16:47:26.317 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T16:47:26.359 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T16:47:26.677 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T16:47:26.719 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T16:47:26.802 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T16:47:27.111 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T16:47:27.270 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T16:47:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T16:47:27.671 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T16:47:27.850 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T16:47:27.982 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T16:47:28.064 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T16:47:28.108 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T16:47:28.450 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T16:47:28.540 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T16:47:28.925 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T16:47:29.127 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-09-13T16:47:29.334 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T16:47:29.503 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T16:47:29.702 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T16:47:29.946 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T16:47:30.078 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T16:47:30.204 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T16:47:30.322 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T16:47:30.398 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T16:47:30.524 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-common (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:30.706 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T16:47:30.900 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:47:31.085 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T16:47:31.233 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-09-13T16:47:31.519 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T16:47:31.634 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T16:47:31.646 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T16:47:31.780 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T16:47:32.085 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T16:47:32.164 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T16:47:32.537 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T16:47:32.566 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:47:32.663 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-ceph-common (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:32.733 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T16:47:32.859 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T16:47:33.131 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:47:33.261 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T16:47:33.273 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T16:47:33.612 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T16:47:33.708 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T16:47:33.834 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T16:47:33.972 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T16:47:33.994 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T16:47:34.098 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T16:47:34.362 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T16:47:34.441 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T16:47:34.763 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:47:34.792 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T16:47:34.889 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T16:47:35.098 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T16:47:35.167 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T16:47:35.550 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T16:47:35.584 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T16:47:35.901 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T16:47:35.920 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T16:47:36.055 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T16:47:36.277 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T16:47:36.403 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T16:47:36.447 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T16:47:36.771 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T16:47:36.882 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T16:47:37.050 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T16:47:37.147 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T16:47:37.176 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T16:47:37.556 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T16:47:37.569 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T16:47:37.893 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T16:47:37.928 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T16:47:38.234 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T16:47:38.360 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T16:47:38.761 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T16:47:39.271 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T16:47:39.388 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T16:47:39.514 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T16:47:39.751 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T16:47:39.877 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T16:47:39.941 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T16:47:40.325 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T16:47:40.345 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T16:47:40.713 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T16:47:41.056 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:47:41.181 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T16:47:41.608 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T16:47:42.043 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T16:47:42.272 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T16:47:42.440 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T16:47:42.569 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T16:47:42.900 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T16:47:42.987 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T16:47:43.276 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T16:47:43.635 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T16:47:43.761 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T16:47:44.196 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T16:47:44.243 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T16:47:44.368 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T16:47:44.495 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T16:47:44.623 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T16:47:44.646 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T16:47:45.005 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T16:47:45.131 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T16:47:45.149 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T16:47:45.499 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T16:47:45.567 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T16:47:45.850 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T16:47:46.234 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T16:47:46.687 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librados2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:46.864 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T16:47:46.940 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T16:47:47.023 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T16:47:47.066 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librgw2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:47.151 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T16:47:47.192 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libsqlite3-mod-ceph (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:47.277 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T16:47:47.343 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:47.469 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T16:47:47.687 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T16:47:47.829 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libradosstriper1 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:47.830 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T16:47:47.955 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T16:47:48.197 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T16:47:48.315 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librbd1 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:48.474 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-modules-core (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:48.573 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T16:47:48.600 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-fuse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:48.852 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T16:47:48.853 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T16:47:48.974 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T16:47:49.435 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librados2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:49.655 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T16:47:49.668 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs-dev (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:49.794 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rados (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:49.814 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librgw2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:49.921 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-top (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:49.940 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libsqlite3-mod-ceph (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:50.088 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T16:47:50.099 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libcephfs2 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:50.225 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T16:47:50.602 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libradosstriper1 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:50.728 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T16:47:51.154 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librbd1 (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:51.226 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rbd (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:51.322 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-modules-core (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:51.351 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up rbd-fuse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:51.448 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-fuse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:51.478 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rgw (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:51.629 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cephfs (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:51.713 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T16:47:51.713 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T16:47:51.755 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-shell (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:51.881 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-common (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:52.552 INFO:teuthology.orchestra.run.smithi157.stdout:Adding group ceph....done 2024-09-13T16:47:52.626 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libcephfs-dev (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:52.777 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rados (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:52.903 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-top (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:53.020 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user ceph....done 2024-09-13T16:47:53.028 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T16:47:53.248 INFO:teuthology.orchestra.run.smithi157.stdout:Setting system user ceph properties....done 2024-09-13T16:47:53.284 INFO:teuthology.orchestra.run.smithi157.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T16:47:53.493 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T16:47:54.014 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T16:47:54.216 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rbd (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:54.367 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up rbd-fuse (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:54.493 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rgw (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:54.619 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cephfs (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:54.778 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-shell (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:54.904 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-common (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:54.907 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-mirror (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:55.166 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:47:55.166 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:47:55.534 INFO:teuthology.orchestra.run.smithi003.stdout:Adding group ceph....done 2024-09-13T16:47:55.992 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-test (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:56.017 INFO:teuthology.orchestra.run.smithi003.stdout:Adding system user ceph....done 2024-09-13T16:47:56.118 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up radosgw (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:56.245 INFO:teuthology.orchestra.run.smithi003.stdout:Setting system user ceph properties....done 2024-09-13T16:47:56.285 INFO:teuthology.orchestra.run.smithi003.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T16:47:56.502 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T16:47:56.871 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:47:56.871 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:47:57.056 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T16:47:57.687 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-base (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:57.963 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-mirror (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:58.223 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:47:58.223 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:47:58.234 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T16:47:59.121 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mds (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:59.140 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-test (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:59.299 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up radosgw (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:47:59.392 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T16:47:59.392 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T16:48:00.015 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:48:00.015 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:48:00.257 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:00.521 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T16:48:00.521 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T16:48:00.859 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-base (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:01.358 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-osd (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:01.384 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T16:48:02.220 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T16:48:02.221 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T16:48:02.319 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mds (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:02.617 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T16:48:02.617 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T16:48:03.059 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-k8sevents (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:03.185 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:03.372 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mon (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:03.533 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:03.632 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T16:48:03.632 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T16:48:03.822 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T16:48:03.823 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T16:48:04.498 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-cephadm (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:04.692 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:04.705 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-osd (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:04.851 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-dashboard (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:05.002 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-volume (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:05.439 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T16:48:05.526 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T16:48:05.526 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T16:48:05.624 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T16:48:06.092 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T16:48:06.374 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-k8sevents (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:06.492 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:06.676 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mon (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:06.926 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T16:48:06.926 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T16:48:07.802 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-cephadm (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:07.954 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:08.079 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-dashboard (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-volume (18.2.4-850-g50bbe7f3-1focal) ... 2024-09-13T16:48:08.734 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T16:48:08.918 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T16:48:09.395 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T16:48:11.095 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T16:48:11.891 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T16:48:11.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T16:48:12.014 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-09-13T16:48:12.234 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2024-09-13T16:48:12.235 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2024-09-13T16:48:12.461 INFO:teuthology.orchestra.run.smithi157.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-09-13T16:48:12.461 INFO:teuthology.orchestra.run.smithi157.stdout:flex is already the newest version (2.6.4-6.2). 2024-09-13T16:48:12.461 INFO:teuthology.orchestra.run.smithi157.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-09-13T16:48:12.462 INFO:teuthology.orchestra.run.smithi157.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-09-13T16:48:12.462 INFO:teuthology.orchestra.run.smithi157.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-09-13T16:48:12.462 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:48:12.462 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T16:48:12.462 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:48:12.462 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2024-09-13T16:48:12.464 INFO:teuthology.orchestra.run.smithi157.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-09-13T16:48:12.464 INFO:teuthology.orchestra.run.smithi157.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-09-13T16:48:12.464 INFO:teuthology.orchestra.run.smithi157.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-09-13T16:48:12.464 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-09-13T16:48:12.464 INFO:teuthology.orchestra.run.smithi157.stdout: usb-modeswitch-data wpasupplicant 2024-09-13T16:48:12.465 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2024-09-13T16:48:12.465 INFO:teuthology.orchestra.run.smithi157.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-09-13T16:48:12.465 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-09-13T16:48:12.465 INFO:teuthology.orchestra.run.smithi157.stdout: libengine-pkcs11-openssl 2024-09-13T16:48:12.574 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2024-09-13T16:48:12.574 INFO:teuthology.orchestra.run.smithi157.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-09-13T16:48:12.574 INFO:teuthology.orchestra.run.smithi157.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-09-13T16:48:12.575 INFO:teuthology.orchestra.run.smithi157.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-09-13T16:48:12.576 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-09-13T16:48:12.576 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-09-13T16:48:12.576 INFO:teuthology.orchestra.run.smithi157.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T16:48:12.576 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages will be upgraded: 2024-09-13T16:48:12.577 INFO:teuthology.orchestra.run.smithi157.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-09-13T16:48:12.794 INFO:teuthology.orchestra.run.smithi157.stdout:6 upgraded, 31 newly installed, 0 to remove and 292 not upgraded. 2024-09-13T16:48:12.794 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 25.4 MB of archives. 2024-09-13T16:48:12.794 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 80.0 MB of additional disk space will be used. 2024-09-13T16:48:12.794 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-09-13T16:48:13.411 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-09-13T16:48:13.413 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-09-13T16:48:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-09-13T16:48:13.427 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-09-13T16:48:13.429 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-09-13T16:48:13.527 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-09-13T16:48:13.529 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-09-13T16:48:13.536 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-09-13T16:48:13.541 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-09-13T16:48:13.542 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-09-13T16:48:13.574 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-09-13T16:48:13.576 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-09-13T16:48:13.580 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-09-13T16:48:13.581 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-09-13T16:48:13.655 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-09-13T16:48:13.655 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-09-13T16:48:13.665 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.20-0ubuntu0.20.04.1 [117 kB] 2024-09-13T16:48:13.668 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-09-13T16:48:13.684 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-09-13T16:48:13.684 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-09-13T16:48:13.736 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-09-13T16:48:13.737 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-09-13T16:48:13.759 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-09-13T16:48:13.789 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-09-13T16:48:13.847 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-09-13T16:48:13.852 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-09-13T16:48:13.853 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-09-13T16:48:13.857 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-09-13T16:48:13.857 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.20-0ubuntu0.20.04.1 [1,055 kB] 2024-09-13T16:48:13.881 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-09-13T16:48:13.899 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.20-0ubuntu0.20.04.1 [13.5 MB] 2024-09-13T16:48:14.288 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-09-13T16:48:14.289 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-09-13T16:48:14.289 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-09-13T16:48:14.292 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-09-13T16:48:14.293 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-09-13T16:48:14.540 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T16:48:14.860 INFO:teuthology.orchestra.run.smithi157.stdout:Preconfiguring packages ... 2024-09-13T16:48:15.095 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 25.4 MB in 2s (14.9 MB/s) 2024-09-13T16:48:15.339 INFO:teuthology.orchestra.run.smithi003.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T16:48:15.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T16:48:15.447 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 119132 files and directories currently installed.) 2024-09-13T16:48:15.451 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T16:48:15.476 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-13T16:48:15.654 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T16:48:15.702 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2024-09-13T16:48:15.703 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2024-09-13T16:48:15.935 INFO:teuthology.orchestra.run.smithi003.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-09-13T16:48:15.936 INFO:teuthology.orchestra.run.smithi003.stdout:flex is already the newest version (2.6.4-6.2). 2024-09-13T16:48:15.936 INFO:teuthology.orchestra.run.smithi003.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-09-13T16:48:15.936 INFO:teuthology.orchestra.run.smithi003.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-09-13T16:48:15.936 INFO:teuthology.orchestra.run.smithi003.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-09-13T16:48:15.936 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:48:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T16:48:15.936 INFO:teuthology.orchestra.run.smithi003.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:48:15.937 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2024-09-13T16:48:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-09-13T16:48:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-09-13T16:48:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-09-13T16:48:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-09-13T16:48:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: usb-modeswitch-data wpasupplicant 2024-09-13T16:48:15.939 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2024-09-13T16:48:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-09-13T16:48:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-09-13T16:48:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: libengine-pkcs11-openssl 2024-09-13T16:48:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2024-09-13T16:48:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-09-13T16:48:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-09-13T16:48:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-09-13T16:48:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-09-13T16:48:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-09-13T16:48:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T16:48:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages will be upgraded: 2024-09-13T16:48:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-09-13T16:48:16.249 INFO:teuthology.orchestra.run.smithi003.stdout:6 upgraded, 31 newly installed, 0 to remove and 292 not upgraded. 2024-09-13T16:48:16.249 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 25.4 MB of archives. 2024-09-13T16:48:16.249 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 80.0 MB of additional disk space will be used. 2024-09-13T16:48:16.249 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-09-13T16:48:16.865 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-09-13T16:48:16.867 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-09-13T16:48:16.867 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-09-13T16:48:16.880 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-09-13T16:48:16.882 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-09-13T16:48:16.980 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-09-13T16:48:16.982 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-09-13T16:48:16.989 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-09-13T16:48:16.994 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-09-13T16:48:16.994 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-09-13T16:48:17.028 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-09-13T16:48:17.029 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-09-13T16:48:17.032 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-09-13T16:48:17.033 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-09-13T16:48:17.109 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-09-13T16:48:17.109 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-09-13T16:48:17.119 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.20-0ubuntu0.20.04.1 [117 kB] 2024-09-13T16:48:17.122 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-09-13T16:48:17.138 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-09-13T16:48:17.138 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-09-13T16:48:17.190 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-09-13T16:48:17.191 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-09-13T16:48:17.214 INFO:teuthology.orchestra.run.smithi003.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-09-13T16:48:17.242 INFO:teuthology.orchestra.run.smithi003.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-09-13T16:48:17.300 INFO:teuthology.orchestra.run.smithi003.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-09-13T16:48:17.308 INFO:teuthology.orchestra.run.smithi003.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-09-13T16:48:17.309 INFO:teuthology.orchestra.run.smithi003.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-09-13T16:48:17.310 INFO:teuthology.orchestra.run.smithi003.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-09-13T16:48:17.310 INFO:teuthology.orchestra.run.smithi003.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.20-0ubuntu0.20.04.1 [1,055 kB] 2024-09-13T16:48:17.330 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-09-13T16:48:17.337 INFO:teuthology.orchestra.run.smithi003.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-09-13T16:48:17.354 INFO:teuthology.orchestra.run.smithi003.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.20-0ubuntu0.20.04.1 [13.5 MB] 2024-09-13T16:48:17.745 INFO:teuthology.orchestra.run.smithi003.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-09-13T16:48:17.745 INFO:teuthology.orchestra.run.smithi003.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-09-13T16:48:17.746 INFO:teuthology.orchestra.run.smithi003.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-09-13T16:48:17.746 INFO:teuthology.orchestra.run.smithi003.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-09-13T16:48:17.746 INFO:teuthology.orchestra.run.smithi003.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-09-13T16:48:18.335 INFO:teuthology.orchestra.run.smithi003.stdout:Preconfiguring packages ... 2024-09-13T16:48:18.602 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 25.4 MB in 2s (15.0 MB/s) 2024-09-13T16:48:18.968 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 119132 files and directories currently installed.) 2024-09-13T16:48:18.972 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T16:48:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T16:48:19.335 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 119132 files and directories currently installed.) 2024-09-13T16:48:19.339 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T16:48:19.463 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T16:48:19.922 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T16:48:20.249 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T16:48:20.634 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T16:48:20.868 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-09-13T16:48:21.132 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 119132 files and directories currently installed.) 2024-09-13T16:48:21.135 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T16:48:21.303 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T16:48:21.827 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T16:48:21.958 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T16:48:22.558 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T16:48:22.688 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-09-13T16:48:22.878 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 119132 files and directories currently installed.) 2024-09-13T16:48:22.881 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T16:48:23.015 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T16:48:23.284 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T16:48:23.302 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-09-13T16:48:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-09-13T16:48:23.521 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T16:48:23.688 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T16:48:23.737 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package dump. 2024-09-13T16:48:23.756 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-09-13T16:48:23.794 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking dump (0.4b46-6) ... 2024-09-13T16:48:24.082 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T16:48:24.173 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package indent. 2024-09-13T16:48:24.193 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:48:24.231 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T16:48:24.576 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 119132 files and directories currently installed.) 2024-09-13T16:48:24.580 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T16:48:24.601 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T16:48:24.623 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-09-13T16:48:24.668 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T16:48:24.746 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T16:48:25.207 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T16:48:25.226 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-09-13T16:48:25.265 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T16:48:25.270 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T16:48:25.435 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T16:48:25.581 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T16:48:25.604 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T16:48:25.651 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T16:48:25.984 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T16:48:26.054 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T16:48:26.075 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:26.112 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T16:48:26.122 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-09-13T16:48:26.432 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T16:48:26.450 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:26.490 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T16:48:26.693 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T16:48:26.716 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-09-13T16:48:26.760 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-09-13T16:48:26.851 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T16:48:26.870 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:26.909 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T16:48:27.171 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package dump. 2024-09-13T16:48:27.191 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-09-13T16:48:27.229 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking dump (0.4b46-6) ... 2024-09-13T16:48:27.329 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T16:48:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-09-13T16:48:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T16:48:27.759 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package indent. 2024-09-13T16:48:27.778 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:48:27.799 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T16:48:27.817 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T16:48:27.818 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T16:48:27.857 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T16:48:28.195 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T16:48:28.216 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-09-13T16:48:28.253 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T16:48:28.296 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T16:48:28.313 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../14-libpq5_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:48:28.359 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:28.657 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T16:48:28.679 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-09-13T16:48:28.732 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T16:48:28.763 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T16:48:28.783 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:28.821 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T16:48:29.060 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T16:48:29.079 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T16:48:29.118 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T16:48:29.224 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T16:48:29.245 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:29.290 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T16:48:29.521 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T16:48:29.545 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:29.587 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T16:48:29.593 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T16:48:29.616 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T16:48:29.660 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T16:48:29.890 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T16:48:29.913 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:29.956 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T16:48:30.221 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T16:48:30.241 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-09-13T16:48:30.321 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-09-13T16:48:30.352 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T16:48:30.371 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:30.426 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T16:48:30.758 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package modemmanager. 2024-09-13T16:48:30.779 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:30.829 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T16:48:30.846 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T16:48:30.850 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-09-13T16:48:30.896 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T16:48:31.290 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T16:48:31.314 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T16:48:31.357 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T16:48:31.522 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T16:48:31.542 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-09-13T16:48:31.580 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T16:48:31.802 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T16:48:31.822 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../14-libpq5_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:48:31.860 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:32.177 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package network-manager. 2024-09-13T16:48:32.197 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T16:48:32.279 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T16:48:32.301 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:32.314 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T16:48:32.354 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T16:48:32.765 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T16:48:32.786 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:32.831 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T16:48:33.109 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ppp. 2024-09-13T16:48:33.128 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-09-13T16:48:33.143 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T16:48:33.163 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T16:48:33.201 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T16:48:33.599 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T16:48:33.763 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T16:48:33.783 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-09-13T16:48:33.854 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-09-13T16:48:34.031 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T16:48:34.053 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-09-13T16:48:34.097 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-09-13T16:48:34.333 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package modemmanager. 2024-09-13T16:48:34.352 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:48:34.411 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T16:48:34.517 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T16:48:34.538 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-09-13T16:48:34.606 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-09-13T16:48:34.945 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T16:48:34.969 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-09-13T16:48:35.045 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-09-13T16:48:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T16:48:35.143 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-09-13T16:48:35.180 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T16:48:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client-12. 2024-09-13T16:48:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../26-postgresql-client-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:48:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:35.844 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package network-manager. 2024-09-13T16:48:35.863 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T16:48:35.952 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T16:48:36.480 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T16:48:36.499 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-09-13T16:48:36.570 INFO:teuthology.orchestra.run.smithi157.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T16:48:36.626 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-09-13T16:48:36.759 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ppp. 2024-09-13T16:48:36.780 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-09-13T16:48:37.170 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-12. 2024-09-13T16:48:37.189 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../28-postgresql-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:48:37.235 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:37.259 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T16:48:37.706 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T16:48:37.727 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-09-13T16:48:37.805 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-09-13T16:48:38.226 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T16:48:38.248 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-09-13T16:48:38.328 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-09-13T16:48:38.662 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T16:48:38.683 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-09-13T16:48:38.728 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-09-13T16:48:38.856 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql. 2024-09-13T16:48:38.877 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-09-13T16:48:38.922 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-09-13T16:48:39.108 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client-12. 2024-09-13T16:48:39.127 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../26-postgresql-client-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:48:39.166 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:39.233 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T16:48:39.255 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-09-13T16:48:39.299 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-09-13T16:48:39.601 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T16:48:39.626 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-09-13T16:48:39.668 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T16:48:39.839 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T16:48:39.859 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-09-13T16:48:39.939 INFO:teuthology.orchestra.run.smithi003.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T16:48:39.954 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T16:48:39.965 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-09-13T16:48:39.975 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-09-13T16:48:40.020 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-09-13T16:48:40.482 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T16:48:40.504 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-09-13T16:48:40.544 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-12. 2024-09-13T16:48:40.558 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../28-postgresql-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T16:48:40.592 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T16:48:40.608 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:40.928 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-09-13T16:48:41.179 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T16:48:41.539 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:41.665 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-09-13T16:48:41.833 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T16:48:41.959 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T16:48:42.263 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql. 2024-09-13T16:48:42.286 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-09-13T16:48:42.329 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-09-13T16:48:42.436 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T16:48:42.436 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T16:48:42.640 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T16:48:42.663 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-09-13T16:48:42.706 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-09-13T16:48:43.009 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T16:48:43.031 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-09-13T16:48:43.075 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T16:48:43.378 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T16:48:43.402 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-09-13T16:48:43.411 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:43.444 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-09-13T16:48:43.881 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T16:48:43.900 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-09-13T16:48:43.989 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T16:48:44.338 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-09-13T16:48:44.553 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T16:48:44.921 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:45.047 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-09-13T16:48:45.215 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T16:48:45.343 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T16:48:45.440 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T16:48:45.813 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T16:48:45.813 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T16:48:46.059 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T16:48:46.185 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-09-13T16:48:46.344 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-09-13T16:48:46.737 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:47.651 INFO:teuthology.orchestra.run.smithi157.stdout:Adding user postgres to group ssl-cert 2024-09-13T16:48:48.023 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:48:48.023 INFO:teuthology.orchestra.run.smithi157.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T16:48:48.498 INFO:teuthology.orchestra.run.smithi157.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T16:48:48.498 INFO:teuthology.orchestra.run.smithi157.stdout:Removing obsolete dictionary files: 2024-09-13T16:48:48.797 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T16:48:48.926 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T16:48:49.056 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T16:48:49.077 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-09-13T16:48:49.195 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-09-13T16:48:50.471 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T16:48:50.542 INFO:teuthology.orchestra.run.smithi003.stdout:Adding user postgres to group ssl-cert 2024-09-13T16:48:50.630 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-09-13T16:48:50.764 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T16:48:50.890 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T16:48:50.908 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:48:50.908 INFO:teuthology.orchestra.run.smithi003.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T16:48:51.398 INFO:teuthology.orchestra.run.smithi003.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T16:48:51.398 INFO:teuthology.orchestra.run.smithi003.stdout:Removing obsolete dictionary files: 2024-09-13T16:48:51.972 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T16:48:52.142 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-09-13T16:48:52.616 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up dump (0.4b46-6) ... 2024-09-13T16:48:52.731 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T16:48:52.797 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T16:48:52.923 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T16:48:53.082 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T16:48:53.209 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T16:48:53.435 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T16:48:53.561 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:53.613 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T16:48:53.780 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-09-13T16:48:53.906 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T16:48:54.032 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T16:48:54.115 INFO:teuthology.orchestra.run.smithi157.stdout:Creating new PostgreSQL cluster 12/main ... 2024-09-13T16:48:54.149 INFO:teuthology.orchestra.run.smithi157.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout:This user must also own the server process. 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout:The default text search configuration will be set to "english". 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout:Data page checksums are disabled. 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:48:54.235 INFO:teuthology.orchestra.run.smithi157.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-09-13T16:48:54.238 INFO:teuthology.orchestra.run.smithi157.stdout:creating subdirectories ... ok 2024-09-13T16:48:54.239 INFO:teuthology.orchestra.run.smithi157.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T16:48:54.275 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default max_connections ... 100 2024-09-13T16:48:54.323 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default shared_buffers ... 128MB 2024-09-13T16:48:54.347 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default time zone ... Etc/UTC 2024-09-13T16:48:54.348 INFO:teuthology.orchestra.run.smithi157.stdout:creating configuration files ... ok 2024-09-13T16:48:54.530 INFO:teuthology.orchestra.run.smithi157.stdout:running bootstrap script ... ok 2024-09-13T16:48:55.142 INFO:teuthology.orchestra.run.smithi157.stdout:performing post-bootstrap initialization ... ok 2024-09-13T16:48:55.317 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-09-13T16:48:55.674 INFO:teuthology.orchestra.run.smithi157.stdout:syncing data to disk ... ok 2024-09-13T16:48:55.674 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:48:55.674 INFO:teuthology.orchestra.run.smithi157.stdout:Success. You can now start the database server using: 2024-09-13T16:48:55.674 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:48:55.675 INFO:teuthology.orchestra.run.smithi157.stdout: pg_ctlcluster 12 main start 2024-09-13T16:48:55.675 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:48:55.771 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up dump (0.4b46-6) ... 2024-09-13T16:48:55.880 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T16:48:55.948 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T16:48:56.107 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T16:48:56.220 INFO:teuthology.orchestra.run.smithi157.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-09-13T16:48:56.220 INFO:teuthology.orchestra.run.smithi157.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-09-13T16:48:56.233 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T16:48:56.359 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T16:48:56.425 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T16:48:56.577 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T16:48:56.703 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T16:48:57.215 INFO:teuthology.orchestra.run.smithi003.stdout:Creating new PostgreSQL cluster 12/main ... 2024-09-13T16:48:57.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout:This user must also own the server process. 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout:The default text search configuration will be set to "english". 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout:Data page checksums are disabled. 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:48:57.318 INFO:teuthology.orchestra.run.smithi003.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-09-13T16:48:57.322 INFO:teuthology.orchestra.run.smithi003.stdout:creating subdirectories ... ok 2024-09-13T16:48:57.322 INFO:teuthology.orchestra.run.smithi003.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T16:48:57.358 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default max_connections ... 100 2024-09-13T16:48:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default shared_buffers ... 128MB 2024-09-13T16:48:57.430 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default time zone ... Etc/UTC 2024-09-13T16:48:57.431 INFO:teuthology.orchestra.run.smithi003.stdout:creating configuration files ... ok 2024-09-13T16:48:57.625 INFO:teuthology.orchestra.run.smithi003.stdout:running bootstrap script ... ok 2024-09-13T16:48:58.278 INFO:teuthology.orchestra.run.smithi003.stdout:performing post-bootstrap initialization ... ok 2024-09-13T16:48:58.841 INFO:teuthology.orchestra.run.smithi003.stdout:syncing data to disk ... ok 2024-09-13T16:48:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:48:58.841 INFO:teuthology.orchestra.run.smithi003.stdout:Success. You can now start the database server using: 2024-09-13T16:48:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:48:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: pg_ctlcluster 12 main start 2024-09-13T16:48:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:48:59.432 INFO:teuthology.orchestra.run.smithi003.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-09-13T16:48:59.432 INFO:teuthology.orchestra.run.smithi003.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-09-13T16:48:59.632 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T16:49:00.035 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T16:49:00.161 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T16:49:00.321 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-09-13T16:49:00.489 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T16:49:00.615 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-09-13T16:49:00.774 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T16:49:01.961 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:01.961 INFO:teuthology.orchestra.run.smithi157.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T16:49:01.961 INFO:teuthology.orchestra.run.smithi157.stdout:which means they are currently configured by ifupdown: 2024-09-13T16:49:01.961 INFO:teuthology.orchestra.run.smithi157.stdout:- enp3s0f1 2024-09-13T16:49:01.961 INFO:teuthology.orchestra.run.smithi157.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T16:49:01.962 INFO:teuthology.orchestra.run.smithi157.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T16:49:01.962 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:02.189 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T16:49:02.715 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T16:49:03.285 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T16:49:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T16:49:03.554 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-09-13T16:49:03.690 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T16:49:03.860 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T16:49:03.990 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-09-13T16:49:04.116 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T16:49:04.660 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T16:49:04.795 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-09-13T16:49:04.969 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T16:49:05.097 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T16:49:05.223 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-09-13T16:49:05.328 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:05.329 INFO:teuthology.orchestra.run.smithi003.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T16:49:05.329 INFO:teuthology.orchestra.run.smithi003.stdout:which means they are currently configured by ifupdown: 2024-09-13T16:49:05.329 INFO:teuthology.orchestra.run.smithi003.stdout:- ens1f0 2024-09-13T16:49:05.329 INFO:teuthology.orchestra.run.smithi003.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T16:49:05.329 INFO:teuthology.orchestra.run.smithi003.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T16:49:05.329 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:05.374 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T16:49:05.500 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T16:49:05.566 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T16:49:05.642 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T16:49:05.989 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T16:49:05.990 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T16:49:06.173 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T16:49:06.980 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-09-13T16:49:07.090 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T16:49:07.115 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T16:49:07.232 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T16:49:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T16:49:07.893 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T16:49:08.103 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T16:49:08.228 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-09-13T16:49:08.396 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T16:49:08.547 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T16:49:08.789 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-09-13T16:49:08.965 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T16:49:09.092 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T16:49:09.243 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T16:49:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T16:49:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T16:49:10.587 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-09-13T16:49:10.821 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T16:49:10.955 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T16:49:11.115 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T16:49:11.777 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T16:49:20.721 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-09-13T16:49:20.830 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-09-13T16:49:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T16:49:22.117 DEBUG:teuthology.parallel:result is None 2024-09-13T16:49:25.104 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-09-13T16:49:25.212 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-09-13T16:49:26.484 INFO:teuthology.orchestra.run.smithi003.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T16:49:26.494 DEBUG:teuthology.parallel:result is None 2024-09-13T16:49:26.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:49:26.651 DEBUG:teuthology.orchestra.run.smithi003:> dpkg-query -W -f '${Version}' ceph 2024-09-13T16:49:26.690 INFO:teuthology.orchestra.run.smithi003.stdout:18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:49:26.691 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:49:26.691 INFO:teuthology.task.install:The correct ceph version 18.2.4-850-g50bbe7f3-1focal is installed. 2024-09-13T16:49:26.693 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:49:26.846 DEBUG:teuthology.orchestra.run.smithi157:> dpkg-query -W -f '${Version}' ceph 2024-09-13T16:49:26.888 INFO:teuthology.orchestra.run.smithi157.stdout:18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:49:26.888 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-850-g50bbe7f3-1focal 2024-09-13T16:49:26.888 INFO:teuthology.task.install:The correct ceph version 18.2.4-850-g50bbe7f3-1focal is installed. 2024-09-13T16:49:26.890 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T16:49:26.890 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:26.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T16:49:26.924 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:49:26.924 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T16:49:26.964 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T16:49:26.964 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:26.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T16:49:27.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T16:49:27.084 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:49:27.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T16:49:27.120 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T16:49:27.196 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T16:49:27.197 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:27.197 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T16:49:27.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T16:49:27.318 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:49:27.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T16:49:27.352 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T16:49:27.427 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T16:49:27.427 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:27.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T16:49:27.465 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T16:49:27.540 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:49:27.540 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T16:49:27.571 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T16:49:27.647 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T16:49:27.731 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T16:49:27.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2024-09-13T16:49:27.737 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2024-09-13T16:49:27.772 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T16:49:27.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T16:49:27.819 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T16:49:27.851 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T16:49:27.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T16:49:27.893 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T16:49:27.938 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-13T16:49:27.938 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-13T16:49:27.938 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-13T16:49:27.938 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:27.939 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T16:49:27.985 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:28.111 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:49:28.111 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T16:49:28.130 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:28.257 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T16:49:28.257 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T16:49:28.258 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832', 'branch': 'wip-rishabh-testing-20240910.095512-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable}', 'email': None, 'first_in_suite': False, 'job_id': '7903832', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.095512-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.095512-reef', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1152, 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'sleep_before_teardown': 0, 'subset': '984/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.095512-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_openfiletable']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:36:41', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T16:49:28.258 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T16:49:28.265 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T16:49:28.284 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T16:49:28.329 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T16:49:28.362 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:28.362 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T16:49:28.426 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T16:49:28.426 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-09-13T16:49:28.486 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T16:49:28.486 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:49:28.486 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 497 Links: 1 2024-09-13T16:49:28.486 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:49:28.487 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-13 16:43:55.933993789 +0000 2024-09-13T16:49:28.487 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-13 16:43:55.582001907 +0000 2024-09-13T16:49:28.487 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-13 16:43:55.582001907 +0000 2024-09-13T16:49:28.487 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-09-13T16:49:28.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T16:49:28.564 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:49:28.564 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:49:28.564 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000576555 s, 888 kB/s 2024-09-13T16:49:28.567 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T16:49:28.630 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-09-13T16:49:28.693 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T16:49:28.693 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:49:28.693 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 504 Links: 1 2024-09-13T16:49:28.693 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:49:28.693 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-13 16:43:56.569979122 +0000 2024-09-13T16:49:28.693 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-13 16:43:56.265986133 +0000 2024-09-13T16:49:28.693 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-13 16:43:56.265986133 +0000 2024-09-13T16:49:28.693 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-09-13T16:49:28.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T16:49:28.770 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:49:28.770 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:49:28.770 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000687644 s, 745 kB/s 2024-09-13T16:49:28.773 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T16:49:28.838 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-09-13T16:49:28.901 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T16:49:28.901 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:49:28.901 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 512 Links: 1 2024-09-13T16:49:28.901 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:49:28.901 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-13 16:43:57.253963348 +0000 2024-09-13T16:49:28.901 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-13 16:43:56.921971004 +0000 2024-09-13T16:49:28.901 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-13 16:43:56.921971004 +0000 2024-09-13T16:49:28.901 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-09-13T16:49:28.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T16:49:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:49:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:49:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000681408 s, 751 kB/s 2024-09-13T16:49:28.979 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T16:49:29.041 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-09-13T16:49:29.102 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T16:49:29.103 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:49:29.103 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 518 Links: 1 2024-09-13T16:49:29.103 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:49:29.103 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-13 16:43:57.581955785 +0000 2024-09-13T16:49:29.103 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-13 16:43:57.577955878 +0000 2024-09-13T16:49:29.104 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-13 16:43:57.577955878 +0000 2024-09-13T16:49:29.104 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-09-13T16:49:29.104 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T16:49:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:49:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:49:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000606803 s, 844 kB/s 2024-09-13T16:49:29.177 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T16:49:29.244 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:29.244 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:49:29.244 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T16:49:29.261 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T16:49:29.261 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-09-13T16:49:29.319 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T16:49:29.320 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:49:29.320 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 498 Links: 1 2024-09-13T16:49:29.320 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:49:29.320 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-13 16:43:51.848061721 +0000 2024-09-13T16:49:29.320 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-13 16:43:51.536068916 +0000 2024-09-13T16:49:29.320 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-13 16:43:51.536068916 +0000 2024-09-13T16:49:29.320 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-09-13T16:49:29.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T16:49:29.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-13T16:49:29.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-13T16:49:29.393 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000605111 s, 846 kB/s 2024-09-13T16:49:29.395 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T16:49:29.462 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-09-13T16:49:29.520 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T16:49:29.520 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:49:29.520 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 503 Links: 1 2024-09-13T16:49:29.520 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:49:29.520 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-13 16:43:52.476047238 +0000 2024-09-13T16:49:29.520 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-13 16:43:52.156054617 +0000 2024-09-13T16:49:29.520 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-13 16:43:52.156054617 +0000 2024-09-13T16:49:29.520 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-09-13T16:49:29.521 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T16:49:29.596 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-13T16:49:29.596 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-13T16:49:29.596 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000580127 s, 883 kB/s 2024-09-13T16:49:29.598 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T16:49:29.662 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-09-13T16:49:29.721 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T16:49:29.721 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:49:29.721 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 511 Links: 1 2024-09-13T16:49:29.722 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:49:29.722 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-13 16:43:53.100032848 +0000 2024-09-13T16:49:29.722 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-13 16:43:52.784040135 +0000 2024-09-13T16:49:29.722 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-13 16:43:52.784040135 +0000 2024-09-13T16:49:29.722 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-09-13T16:49:29.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T16:49:29.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-13T16:49:29.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-13T16:49:29.793 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000470535 s, 1.1 MB/s 2024-09-13T16:49:29.795 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T16:49:29.859 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-09-13T16:49:29.926 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T16:49:29.927 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:49:29.927 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 518 Links: 1 2024-09-13T16:49:29.927 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:49:29.927 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-13 16:43:53.384026299 +0000 2024-09-13T16:49:29.927 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-13 16:43:53.380026390 +0000 2024-09-13T16:49:29.927 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-13 16:43:53.380026390 +0000 2024-09-13T16:49:29.927 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-09-13T16:49:29.927 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T16:49:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-13T16:49:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-13T16:49:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000414294 s, 1.2 MB/s 2024-09-13T16:49:30.001 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T16:49:30.068 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:30.068 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T16:49:30.068 INFO:tasks.ceph:Generating config... 2024-09-13T16:49:30.069 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T16:49:30.069 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T16:49:30.069 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T16:49:30.070 INFO:tasks.ceph:[client] pid file = 2024-09-13T16:49:30.070 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T16:49:30.070 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T16:49:30.070 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T16:49:30.070 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T16:49:30.071 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T16:49:30.071 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T16:49:30.071 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T16:49:30.071 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-13T16:49:30.071 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-13T16:49:30.071 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T16:49:30.071 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T16:49:30.072 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-13T16:49:30.072 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T16:49:30.072 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T16:49:30.072 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T16:49:30.072 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T16:49:30.072 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T16:49:30.072 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T16:49:30.072 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T16:49:30.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T16:49:30.148 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T16:49:30.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T16:49:30.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T16:49:30.271 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.b', '172.21.15.157'), ('mon.c', '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]')] 2024-09-13T16:49:30.271 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T16:49:30.272 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:30.272 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T16:49:30.333 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --add b 172.21.15.157 --addv c '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:30.447 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = pacific 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2024-09-13T16:49:30.445947+0000 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-09-13T16:49:30.445947+0000 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 16 (pacific) 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.b 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] mon.c 2024-09-13T16:49:30.448 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T16:49:30.451 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T16:49:30.467 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 61456116-a42d-4065-9bf5-a1daf0be594e... 2024-09-13T16:49:30.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:49:30.517 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:49:30.587 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-13T16:49:30.588 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-13T16:49:30.588 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:49:30.588 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-13T16:49:30.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.588 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-13T16:49:30.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.588 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-13T16:49:30.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.590 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:49:30.591 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.592 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-13T16:49:30.593 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:49:30.594 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-13T16:49:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:49:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:49:30.597 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-13T16:49:30.598 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:49:30.599 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.600 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-09-13T16:49:30.601 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.602 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.603 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-09-13T16:49:30.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-09-13T16:49:30.605 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:49:30.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-09-13T16:49:30.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-09-13T16:49:30.608 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:49:30.609 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.609 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:49:30.609 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-09-13T16:49:30.609 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on pool no app = False 2024-09-13T16:49:30.609 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:49:30.609 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-09-13T16:49:30.609 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-09-13T16:49:30.609 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-09-13T16:49:30.610 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-09-13T16:49:30.611 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:49:30.612 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:49:30.612 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:49:30.613 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T16:49:30.613 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T16:49:30.707 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T16:49:30.707 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:30.707 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T16:49:30.720 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:30.720 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T16:49:30.782 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:49:30.782 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:30.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T16:49:30.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T16:49:30.883 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:49:30.883 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:30.941 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:49:30.942 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:49:30.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T16:49:30.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T16:49:30.993 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:49:30.994 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:31.051 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T16:49:31.052 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T16:49:31.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T16:49:31.148 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T16:49:31.156 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T16:49:31.252 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T16:49:31.272 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T16:49:31.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T16:49:31.372 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T16:49:31.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T16:49:31.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T16:49:31.555 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T16:49:31.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-13T16:49:31.600 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T16:49:31.704 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T16:49:31.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-13T16:49:31.746 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T16:49:31.881 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T16:49:31.887 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-13T16:49:31.913 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T16:49:31.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T16:49:31.987 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T16:49:32.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-13T16:49:32.093 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-13T16:49:32.123 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T16:49:32.191 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T16:49:32.220 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-13T16:49:32.284 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-13T16:49:32.313 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T16:49:32.314 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T16:49:32.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T16:49:32.351 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:32.351 INFO:tasks.ceph:role: osd.0 2024-09-13T16:49:32.351 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:49:32.351 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T16:49:32.936 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:49:32.936 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:49:32.936 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:49:32.936 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2024-09-13T16:49:32.936 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:49:32.937 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:49:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:49:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:49:32.937 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:49:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:49:32.941 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-13T16:49:32.942 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T16:49:33.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T16:49:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:49:33.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:49:33.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T16:49:33.157 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:33.157 INFO:tasks.ceph:role: osd.1 2024-09-13T16:49:33.158 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:49:33.160 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:49:33.707 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:49:33.713 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-13T16:49:33.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T16:49:33.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T16:49:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:49:33.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:49:33.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T16:49:33.877 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:33.877 INFO:tasks.ceph:role: osd.2 2024-09-13T16:49:33.877 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:49:33.878 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:49:34.402 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:49:34.407 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-13T16:49:34.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T16:49:34.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T16:49:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:49:34.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:49:34.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T16:49:34.635 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:34.635 INFO:tasks.ceph:role: osd.3 2024-09-13T16:49:34.635 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:49:34.635 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T16:49:35.204 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:49:35.204 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:49:35.204 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:49:35.205 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2024-09-13T16:49:35.205 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:49:35.205 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:49:35.205 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:49:35.205 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:49:35.205 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:49:35.205 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:49:35.208 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-13T16:49:35.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T16:49:35.420 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T16:49:35.442 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:49:35.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:49:35.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:35.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:35.605+0000 7f89108ec880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T16:49:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:35.609+0000 7f89108ec880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T16:49:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:35.609+0000 7f89108ec880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T16:49:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:35.609+0000 7f89108ec880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T16:49:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:35.609+0000 7f89108ec880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T16:49:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:35.613+0000 7f89108ec880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T16:49:38.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T16:49:38.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:38.865+0000 7fc5df2e8880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T16:49:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:38.865+0000 7fc5df2e8880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T16:49:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:38.865+0000 7fc5df2e8880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T16:49:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:38.865+0000 7fc5df2e8880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T16:49:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:38.865+0000 7fc5df2e8880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T16:49:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:38.865+0000 7fc5df2e8880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T16:49:41.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T16:49:42.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:42.145+0000 7f85e3767880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T16:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:42.145+0000 7f85e3767880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T16:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:42.145+0000 7f85e3767880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T16:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:42.145+0000 7f85e3767880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T16:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:42.145+0000 7f85e3767880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T16:49:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:42.149+0000 7f85e3767880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T16:49:45.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T16:49:45.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:45.405+0000 7f64b878c880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T16:49:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:45.405+0000 7f64b878c880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T16:49:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:45.405+0000 7f64b878c880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T16:49:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:45.405+0000 7f64b878c880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T16:49:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:45.405+0000 7f64b878c880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T16:49:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:49:45.409+0000 7f64b878c880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T16:49:48.528 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T16:49:48.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T16:49:48.601 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:48.601 INFO:tasks.ceph:role: osd.4 2024-09-13T16:49:48.602 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:49:48.602 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T16:49:48.711 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:49:48.712 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:49:48.721 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-09-13T16:49:48.721 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T16:49:48.765 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T16:49:48.835 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:49:48.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:49:48.837 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T16:49:48.912 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:48.913 INFO:tasks.ceph:role: osd.5 2024-09-13T16:49:48.913 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:49:48.913 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:49:49.023 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:49:49.027 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-09-13T16:49:49.028 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T16:49:49.062 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T16:49:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:49:49.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:49:49.129 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T16:49:49.206 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:49.206 INFO:tasks.ceph:role: osd.6 2024-09-13T16:49:49.206 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:49:49.206 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:49:49.317 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:49:49.321 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-09-13T16:49:49.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T16:49:49.360 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T16:49:49.431 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:49:49.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:49:49.432 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T16:49:49.511 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:49:49.511 INFO:tasks.ceph:role: osd.7 2024-09-13T16:49:49.511 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:49:49.512 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T16:49:49.628 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:49:49.628 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:49:49.628 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:49:49.628 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 2024-09-13T16:49:49.628 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:49:49.628 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:49:49.629 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:49:49.629 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:49:49.629 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:49:49.629 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:49:49.631 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-09-13T16:49:49.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T16:49:49.670 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T16:49:49.738 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:49:49.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:49:49.739 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:49.901+0000 7f0a98cc2880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T16:49:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:49.901+0000 7f0a98cc2880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T16:49:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:49.901+0000 7f0a98cc2880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T16:49:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:49.901+0000 7f0a98cc2880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T16:49:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:49.905+0000 7f0a98cc2880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T16:49:49.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:49.909+0000 7f0a98cc2880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T16:49:53.011 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T16:49:53.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:53.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:53.165+0000 7f9f91a03880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T16:49:53.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:53.169+0000 7f9f91a03880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T16:49:53.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:53.169+0000 7f9f91a03880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T16:49:53.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:53.169+0000 7f9f91a03880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T16:49:53.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:53.169+0000 7f9f91a03880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T16:49:53.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:53.169+0000 7f9f91a03880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T16:49:56.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T16:49:56.271 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:56.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:56.393+0000 7f9005405880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T16:49:56.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:56.393+0000 7f9005405880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T16:49:56.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:56.393+0000 7f9005405880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T16:49:56.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:56.393+0000 7f9005405880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T16:49:56.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:56.393+0000 7f9005405880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T16:49:56.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:56.397+0000 7f9005405880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T16:49:58.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:58.265 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:59.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T16:49:59.517 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:49:59.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:59.640+0000 7f808c649880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T16:49:59.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:59.640+0000 7f808c649880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T16:49:59.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:59.640+0000 7f808c649880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T16:49:59.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:59.640+0000 7f808c649880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T16:49:59.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:59.640+0000 7f808c649880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T16:49:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:49:59.644+0000 7f808c649880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T16:50:02.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T16:50:02.769 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T16:50:02.769 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:02.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T16:50:02.805 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:02.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T16:50:02.885 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:02.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-13T16:50:02.960 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:02.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T16:50:03.039 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:03.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T16:50:03.112 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:03.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T16:50:03.187 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:03.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T16:50:03.259 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.259 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T16:50:03.290 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.291 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-13T16:50:03.373 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.373 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-13T16:50:03.454 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T16:50:03.525 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.525 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T16:50:03.601 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.602 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T16:50:03.676 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.676 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T16:50:03.760 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:03.761 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T16:50:03.780 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:03.780 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-13T16:50:03.839 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.839 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T16:50:03.857 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:03.857 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-13T16:50:03.915 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T16:50:03.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T16:50:03.922 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T16:50:03.965 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.y] 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAbbeRmpP++CBAAYq8DHAVQrKZUzQMWAjwOQA== 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAbbeRm7PYhFhAAjGhG4+MGhybgDuUm80b3GA== 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAbbeRmv2QBIRAAFCBAGuVsMDrEEbFHboF5fA== 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAfbeRmtMg4JBAAhpD6J8l0s10MToeuFpzBnQ== 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAibeRmxu+sMxAAdKOLbj/zALwuUdqEcMGKOA== 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAmbeRmneXPCBAA+5aq4baFkLLf9YgyT1Ej7A== 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2024-09-13T16:50:03.966 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQApbeRmEJ5PGBAAm4ESpLC8pDcykDWH61UX+Q== 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAbbeRmEAD+DhAAwiQnsaTbFldMIXOfTHEdow== 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAbbeRmwv/tKRAAkP5eDK1/mzVXdvg14IIyBQ== 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAbbeRmqBN/NBAAcrl9JO1UMHIeJVdkAmMzBQ== 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.4] 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAtbeRmJ1v/NRAAVnN6OX537lXr9C2U5OwVcg== 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.5] 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAxbeRm1IErChAAlZJr0hgRM7hMjV9IKbFn5Q== 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.6] 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA0beRmjcW0FxAA8jqbdFW19Enk37M00fT5Mg== 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.7] 2024-09-13T16:50:03.967 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA3beRmEURxJhAAbjA9Q82Jn0eyhkHLV3DnXg== 2024-09-13T16:50:03.968 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-13T16:50:03.968 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAbbeRm7UDIOhAAQ85WuAGFt3D40/vdHCx/SQ== 2024-09-13T16:50:03.968 INFO:teuthology.orchestra.run.smithi003.stdout:[client.2] 2024-09-13T16:50:03.968 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAcbeRmpROABRAA7JM6b7+quVMQ4WAlZ5Z0Kg== 2024-09-13T16:50:03.968 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2024-09-13T16:50:03.968 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAcbeRm869WCxAALRQBL9frcnCHq6I4dwzKOA== 2024-09-13T16:50:03.968 INFO:teuthology.orchestra.run.smithi003.stdout:[client.3] 2024-09-13T16:50:03.968 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAcbeRmYTniEBAALAPqutUd7WDZ6lUN3Zk4iQ== 2024-09-13T16:50:03.989 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbbeRmpP++CBAAYq8DHAVQrKZUzQMWAjwOQA== 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbbeRm7PYhFhAAjGhG4+MGhybgDuUm80b3GA== 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbbeRmv2QBIRAAFCBAGuVsMDrEEbFHboF5fA== 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAfbeRmtMg4JBAAhpD6J8l0s10MToeuFpzBnQ== 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAibeRmxu+sMxAAdKOLbj/zALwuUdqEcMGKOA== 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2024-09-13T16:50:03.990 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAmbeRmneXPCBAA+5aq4baFkLLf9YgyT1Ej7A== 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQApbeRmEJ5PGBAAm4ESpLC8pDcykDWH61UX+Q== 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbbeRmEAD+DhAAwiQnsaTbFldMIXOfTHEdow== 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbbeRmwv/tKRAAkP5eDK1/mzVXdvg14IIyBQ== 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbbeRmqBN/NBAAcrl9JO1UMHIeJVdkAmMzBQ== 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAtbeRmJ1v/NRAAVnN6OX537lXr9C2U5OwVcg== 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAxbeRm1IErChAAlZJr0hgRM7hMjV9IKbFn5Q== 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2024-09-13T16:50:03.991 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA0beRmjcW0FxAA8jqbdFW19Enk37M00fT5Mg== 2024-09-13T16:50:03.992 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2024-09-13T16:50:03.992 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA3beRmEURxJhAAbjA9Q82Jn0eyhkHLV3DnXg== 2024-09-13T16:50:03.992 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2024-09-13T16:50:03.992 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbbeRm7UDIOhAAQ85WuAGFt3D40/vdHCx/SQ== 2024-09-13T16:50:03.992 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2024-09-13T16:50:03.992 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAcbeRmpROABRAA7JM6b7+quVMQ4WAlZ5Z0Kg== 2024-09-13T16:50:03.993 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-09-13T16:50:03.993 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAcbeRm869WCxAALRQBL9frcnCHq6I4dwzKOA== 2024-09-13T16:50:03.993 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2024-09-13T16:50:03.993 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAcbeRmYTniEBAALAPqutUd7WDZ6lUN3Zk4iQ== 2024-09-13T16:50:03.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T16:50:04.017 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T16:50:04.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:50:04.123 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:50:04.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:50:04.214 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:50:04.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.301 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.382 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.468 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.554 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.637 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T16:50:04.643 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T16:50:04.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:50:04.731 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:50:04.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:50:04.820 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:50:04.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.988 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:04.994 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:05.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:05.082 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:05.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:05.167 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:50:05.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:50:05.255 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:50:05.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:50:05.346 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:50:05.432 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:50:05.437 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:50:05.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:50:05.527 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:50:05.612 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T16:50:05.612 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T16:50:05.648 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T16:50:05.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T16:50:05.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T16:50:05.862 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T16:50:06.005 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T16:50:06.041 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T16:50:06.115 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T16:50:06.260 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T16:50:06.291 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:50:06.297 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:50:06.353 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T16:50:06.354 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T16:50:06.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T16:50:06.359 INFO:tasks.ceph.mon.a:Started 2024-09-13T16:50:06.359 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T16:50:06.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T16:50:06.400 INFO:tasks.ceph.mon.b:Started 2024-09-13T16:50:06.400 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T16:50:06.401 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T16:50:06.406 INFO:tasks.ceph.mon.c:Started 2024-09-13T16:50:06.406 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T16:50:06.407 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T16:50:06.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T16:50:06.412 INFO:tasks.ceph.mgr.y:Started 2024-09-13T16:50:06.413 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T16:50:06.413 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T16:50:06.416 INFO:tasks.ceph.mgr.x:Started 2024-09-13T16:50:06.416 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T16:50:06.416 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T16:50:06.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7e00700 1 Processor -- start 2024-09-13T16:50:06.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7e00700 1 -- start start 2024-09-13T16:50:06.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6d80ae110 0x7fa6d80668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:06.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6d8163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:06.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d806a390 con 0x7fa6d815a200 2024-09-13T16:50:06.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d8066df0 con 0x7fa6d80ae110 2024-09-13T16:50:06.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8150ee0 con 0x7fa6d8161530 2024-09-13T16:50:06.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d6000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6d80ae110 0x7fa6d80668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:06.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6d8163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:06.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d6a00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa6d815a200 0x7fa6d815a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:33910/0 (socket says 172.21.15.3:33910) 2024-09-13T16:50:06.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d6a00700 1 -- 172.21.15.3:0/2605202931 learned_addr learned my addr 172.21.15.3:0/2605202931 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d6000700 1 -- 172.21.15.3:0/2605202931 >> v2:172.21.15.3:3300/0 conn(0x7fa6d80ae110 msgr2=0x7fa6d80668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T16:50:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d6000700 1 -- 172.21.15.3:0/2605202931 >> v2:172.21.15.3:3300/0 conn(0x7fa6d80ae110 msgr2=0x7fa6d80668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T16:50:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d6000700 1 --2- 172.21.15.3:0/2605202931 >> v2:172.21.15.3:3300/0 conn(0x7fa6d80ae110 0x7fa6d80668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:50:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7400700 1 -- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 msgr2=0x7fa6d8163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T16:50:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7400700 1 -- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 msgr2=0x7fa6d8163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T16:50:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7400700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6d8163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:50:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d6000700 1 --2- 172.21.15.3:0/2605202931 >> v2:172.21.15.3:3300/0 conn(0x7fa6d80ae110 0x7fa6d80668b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:50:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.585+0000 7fa6d7400700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6d8163920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:50:06.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.761+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494527542 0 0) 0x7fa6d806a390 con 0x7fa6d815a200 2024-09-13T16:50:06.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.761+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6bc0035d0 con 0x7fa6d815a200 2024-09-13T16:50:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.761+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 168965519 0 0) 0x7fa6bc0035d0 con 0x7fa6d815a200 2024-09-13T16:50:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.761+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 msgr2=0x7fa6d8163920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T16:50:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.761+0000 7fa6d5600700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6d8163920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.761+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 >> v2:172.21.15.3:3300/0 conn(0x7fa6d80ae110 msgr2=0x7fa6d80668b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T16:50:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.761+0000 7fa6d5600700 1 --2- 172.21.15.3:0/2605202931 >> v2:172.21.15.3:3300/0 conn(0x7fa6d80ae110 0x7fa6d80668b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d806a950 con 0x7fa6d815a200 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fa6c8001670 con 0x7fa6d815a200 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 >> v1:172.21.15.157:6789/0 conn(0x7fa6d815a200 legacy=0x7fa6d815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6bc003a20 0x7fa6bc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6bc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6bc004db0 0x7fa6bc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8066df0 con 0x7fa6bc003a20 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8150ee0 con 0x7fa6bc004db0 2024-09-13T16:50:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc00dda0 con 0x7fa6d8161530 2024-09-13T16:50:06.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d7400700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6bc004db0 0x7fa6bc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:06.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d6000700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6bc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:06.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d6a00700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6bc003a20 0x7fa6bc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:06.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d6000700 1 -- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6bc004db0 msgr2=0x7fa6bc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:06.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d6000700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6bc004db0 0x7fa6bc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d6000700 1 -- 172.21.15.3:0/2605202931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6bc003a20 msgr2=0x7fa6bc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:06.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d6000700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6bc003a20 0x7fa6bc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d6000700 1 -- 172.21.15.3:0/2605202931 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc0035d0 con 0x7fa6d8161530 2024-09-13T16:50:06.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.765+0000 7fa6d6000700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6bc004870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009870 tx=0x7fa6bc0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733818e002f38f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:06.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400c070 con 0x7fa6d8161530 2024-09-13T16:50:06.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c4013610 con 0x7fa6d8161530 2024-09-13T16:50:06.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d5600700 1 -- 172.21.15.3:0/2605202931 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40139c0 con 0x7fa6d8161530 2024-09-13T16:50:06.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d7e00700 1 -- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 msgr2=0x7fa6bc004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:06.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d7e00700 1 --2- 172.21.15.3:0/2605202931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d8161530 0x7fa6bc004870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009870 tx=0x7fa6bc0058a0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d7400700 1 -- 172.21.15.3:0/2605202931 reap_dead start 2024-09-13T16:50:06.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d7e00700 1 -- 172.21.15.3:0/2605202931 shutdown_connections 2024-09-13T16:50:06.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d7e00700 1 -- 172.21.15.3:0/2605202931 >> 172.21.15.3:0/2605202931 conn(0x7fa6d80b23d0 msgr2=0x7fa6d80b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:06.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d7e00700 1 -- 172.21.15.3:0/2605202931 shutdown_connections 2024-09-13T16:50:06.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.769+0000 7fa6d7e00700 1 -- 172.21.15.3:0/2605202931 wait complete. 2024-09-13T16:50:06.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7e00700 1 Processor -- start 2024-09-13T16:50:06.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7e00700 1 -- start start 2024-09-13T16:50:06.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d8161530 0x7fa6d814c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:06.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d80b0350 0x7fa6d80b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:06.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6d815faa0 0x7fa6d815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:06.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8161380 con 0x7fa6d815faa0 2024-09-13T16:50:06.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80b0d10 con 0x7fa6d8161530 2024-09-13T16:50:06.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8160800 con 0x7fa6d80b0350 2024-09-13T16:50:06.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d8161530 0x7fa6d814c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:06.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d80b0350 0x7fa6d80b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d8161530 0x7fa6d814c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:55846/0 (socket says 172.21.15.3:55846) 2024-09-13T16:50:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d6a00700 1 -- 172.21.15.3:0/4234869461 learned_addr learned my addr 172.21.15.3:0/4234869461 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.773+0000 7fa6d7400700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6d815faa0 0x7fa6d815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d6a00700 1 -- 172.21.15.3:0/4234869461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d80b0350 msgr2=0x7fa6d80b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d6a00700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d80b0350 0x7fa6d80b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d6a00700 1 -- 172.21.15.3:0/4234869461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6d815faa0 msgr2=0x7fa6d815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d6a00700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6d815faa0 0x7fa6d815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d6a00700 1 -- 172.21.15.3:0/4234869461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8160ae0 con 0x7fa6d8161530 2024-09-13T16:50:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d6000700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d80b0350 0x7fa6d80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d7400700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6d815faa0 0x7fa6d815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d6a00700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d8161530 0x7fa6d814c390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8007fd0 tx=0x7fa6c80075d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6cea00700 1 -- 172.21.15.3:0/4234869461 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800f6e0 con 0x7fa6d8161530 2024-09-13T16:50:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6cea00700 1 -- 172.21.15.3:0/4234869461 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c800f880 con 0x7fa6d8161530 2024-09-13T16:50:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6cea00700 1 -- 172.21.15.3:0/4234869461 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800fc30 con 0x7fa6d8161530 2024-09-13T16:50:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d8160dc0 con 0x7fa6d8161530 2024-09-13T16:50:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d814c8d0 con 0x7fa6d8161530 2024-09-13T16:50:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6cea00700 1 -- 172.21.15.3:0/4234869461 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa6c800fd90 con 0x7fa6d8161530 2024-09-13T16:50:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6cea00700 1 -- 172.21.15.3:0/4234869461 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa6c8013640 con 0x7fa6d8161530 2024-09-13T16:50:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.777+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d80b3190 con 0x7fa6d8161530 2024-09-13T16:50:06.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.785+0000 7fa6cea00700 1 -- 172.21.15.3:0/4234869461 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fa6c801d8f0 con 0x7fa6d8161530 2024-09-13T16:50:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.861+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa6d806de90 con 0x7fa6d8161530 2024-09-13T16:50:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.869+0000 7fa6cea00700 1 -- 172.21.15.3:0/4234869461 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa6c801da90 con 0x7fa6d8161530 2024-09-13T16:50:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.869+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d8161530 msgr2=0x7fa6d814c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.869+0000 7fa6d7e00700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d8161530 0x7fa6d814c390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8007fd0 tx=0x7fa6c80075d0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.873+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 shutdown_connections 2024-09-13T16:50:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.873+0000 7fa6d7e00700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6d815faa0 0x7fa6d815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.873+0000 7fa6d7e00700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d80b0350 0x7fa6d80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.873+0000 7fa6d7e00700 1 --2- 172.21.15.3:0/4234869461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d8161530 0x7fa6d814c390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.873+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 >> 172.21.15.3:0/4234869461 conn(0x7fa6d80b23d0 msgr2=0x7fa6d814fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.873+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 shutdown_connections 2024-09-13T16:50:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:06.873+0000 7fa6d7e00700 1 -- 172.21.15.3:0/4234869461 wait complete. 2024-09-13T16:50:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-09-13T16:50:06.901 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T16:50:06.901 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T16:50:06.901 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:06.901 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T16:50:06.933 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:06.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T16:50:07.007 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:07.008 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T16:50:07.012 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:07.008+0000 7f8ef30d62c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:50:07.032 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:07.025+0000 7fc3872eb2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:50:07.063 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:07.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T16:50:07.135 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:07.135 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T16:50:07.170 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:07.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T16:50:07.246 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:07.246 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T16:50:07.318 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:50:07.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T16:50:07.389 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new d1db2a65-68c9-405c-bbfa-f9ee9a87343e 0 2024-09-13T16:50:07.597 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.592+0000 7f1398a00700 1 Processor -- start 2024-09-13T16:50:07.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.592+0000 7f1398a00700 1 -- start start 2024-09-13T16:50:07.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f13940cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f139415b110 0x7f139415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13940b41e0 con 0x7f13941442c0 2024-09-13T16:50:07.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f139406a390 con 0x7f139415b110 2024-09-13T16:50:07.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139406b1f0 con 0x7f13940cd7c0 2024-09-13T16:50:07.601 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f13940cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.601 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f13940cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49428/0 (socket says 172.21.15.157:49428) 2024-09-13T16:50:07.601 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392a00700 1 -- 172.21.15.157:0/3011691498 learned_addr learned my addr 172.21.15.157:0/3011691498 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1393e00700 1 --2- 172.21.15.157:0/3011691498 >> v2:172.21.15.3:3300/0 conn(0x7f139415b110 0x7f139415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392a00700 1 -- 172.21.15.157:0/3011691498 >> v2:172.21.15.3:3300/0 conn(0x7f139415b110 msgr2=0x7f139415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392a00700 1 --2- 172.21.15.157:0/3011691498 >> v2:172.21.15.3:3300/0 conn(0x7f139415b110 0x7f139415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392a00700 1 -- 172.21.15.157:0/3011691498 >> v1:172.21.15.157:6789/0 conn(0x7f13941442c0 legacy=0x7f139406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392a00700 1 -- 172.21.15.157:0/3011691498 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13940ce140 con 0x7f13940cd7c0 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392a00700 1 --2- 172.21.15.157:0/3011691498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f13940cdc00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1380009e50 tx=0x7f138000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9550c0d97c4364a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392000700 1 -- 172.21.15.157:0/3011691498 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000e070 con 0x7f13940cd7c0 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392000700 1 -- 172.21.15.157:0/3011691498 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138000b9c0 con 0x7f13940cd7c0 2024-09-13T16:50:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1392000700 1 -- 172.21.15.157:0/3011691498 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000bd70 con 0x7f13940cd7c0 2024-09-13T16:50:07.603 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 -- 172.21.15.157:0/3011691498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 msgr2=0x7f13940cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 --2- 172.21.15.157:0/3011691498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f13940cdc00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1380009e50 tx=0x7f138000b330 comp rx=0 tx=0).stop 2024-09-13T16:50:07.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 -- 172.21.15.157:0/3011691498 shutdown_connections 2024-09-13T16:50:07.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 --2- 172.21.15.157:0/3011691498 >> v2:172.21.15.3:3300/0 conn(0x7f139415b110 0x7f139415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 --2- 172.21.15.157:0/3011691498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f13940cdc00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 -- 172.21.15.157:0/3011691498 >> 172.21.15.157:0/3011691498 conn(0x7f13940af620 msgr2=0x7f13940afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:07.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 -- 172.21.15.157:0/3011691498 shutdown_connections 2024-09-13T16:50:07.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.596+0000 7f1398a00700 1 -- 172.21.15.157:0/3011691498 wait complete. 2024-09-13T16:50:07.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.600+0000 7f1398a00700 1 Processor -- start 2024-09-13T16:50:07.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.600+0000 7f1398a00700 1 -- start start 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.600+0000 7f1398a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f1394159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.600+0000 7f1398a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13941442c0 0x7f1394142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.600+0000 7f1398a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139415b110 0x7f139413d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.600+0000 7f1398a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394142110 con 0x7f13941442c0 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.600+0000 7f1398a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941411b0 con 0x7f139415b110 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.600+0000 7f1398a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1394141490 con 0x7f13940cd7c0 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f1394159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139415b110 0x7f139413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f1394159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49430/0 (socket says 172.21.15.157:49430) 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 -- 172.21.15.157:0/2453025582 learned_addr learned my addr 172.21.15.157:0/2453025582 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1392a00700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13941442c0 0x7f1394142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 -- 172.21.15.157:0/2453025582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139415b110 msgr2=0x7f139413d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139415b110 0x7f139413d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 -- 172.21.15.157:0/2453025582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13941442c0 msgr2=0x7f1394142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13941442c0 0x7f1394142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 -- 172.21.15.157:0/2453025582 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1380009b30 con 0x7f13940cd7c0 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393e00700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139415b110 0x7f139413d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1393400700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f1394159ea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f138400c500 tx=0x7f138400c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f138b400700 1 -- 172.21.15.157:0/2453025582 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840052d0 con 0x7f13940cd7c0 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f138b400700 1 -- 172.21.15.157:0/2453025582 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1384005430 con 0x7f13940cd7c0 2024-09-13T16:50:07.610 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f138b400700 1 -- 172.21.15.157:0/2453025582 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840055a0 con 0x7f13940cd7c0 2024-09-13T16:50:07.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1394141770 con 0x7f13940cd7c0 2024-09-13T16:50:07.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1394141d30 con 0x7f13940cd7c0 2024-09-13T16:50:07.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f138b400700 1 -- 172.21.15.157:0/2453025582 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1384003be0 con 0x7f13940cd7c0 2024-09-13T16:50:07.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f138b400700 1 -- 172.21.15.157:0/2453025582 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f138401c780 con 0x7f13940cd7c0 2024-09-13T16:50:07.611 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.604+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13941446a0 con 0x7f13940cd7c0 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.612+0000 7f138b400700 1 -- 172.21.15.157:0/2453025582 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f1384017070 con 0x7f13940cd7c0 2024-09-13T16:50:07.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.688+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d1db2a65-68c9-405c-bbfa-f9ee9a87343e", "id": 0} v 0) v1 -- 0x7f139415e300 con 0x7f13940cd7c0 2024-09-13T16:50:07.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f138b400700 1 -- 172.21.15.157:0/2453025582 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1db2a65-68c9-405c-bbfa-f9ee9a87343e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1384011d60 con 0x7f13940cd7c0 2024-09-13T16:50:07.702 INFO:teuthology.orchestra.run.smithi157.stdout:0 2024-09-13T16:50:07.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 msgr2=0x7f1394159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f1394159ea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f138400c500 tx=0x7f138400c810 comp rx=0 tx=0).stop 2024-09-13T16:50:07.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 shutdown_connections 2024-09-13T16:50:07.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139415b110 0x7f139413d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13941442c0 0x7f1394142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 --2- 172.21.15.157:0/2453025582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13940cd7c0 0x7f1394159ea0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 >> 172.21.15.157:0/2453025582 conn(0x7f13940af620 msgr2=0x7f139415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:07.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 shutdown_connections 2024-09-13T16:50:07.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.696+0000 7f1398a00700 1 -- 172.21.15.157:0/2453025582 wait complete. 2024-09-13T16:50:07.721 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 02190853-86bc-48c1-b718-2b150b722955 1 2024-09-13T16:50:07.866 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:07.860+0000 7f8ef30d62c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:50:07.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.880+0000 7fd07ce00700 1 Processor -- start 2024-09-13T16:50:07.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.880+0000 7fd07ce00700 1 -- start start 2024-09-13T16:50:07.886 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:07.881+0000 7fc3872eb2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:50:07.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd07815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780b5860 con 0x7fd078157480 2024-09-13T16:50:07.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07806d4b0 con 0x7fd07815e9f0 2024-09-13T16:50:07.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd078157860 con 0x7fd0780b4b40 2024-09-13T16:50:07.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd075400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd07815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd075400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd07815e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49444/0 (socket says 172.21.15.157:49444) 2024-09-13T16:50:07.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd075400700 1 -- 172.21.15.157:0/510049951 learned_addr learned my addr 172.21.15.157:0/510049951 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd075400700 1 -- 172.21.15.157:0/510049951 >> v1:172.21.15.157:6789/0 conn(0x7fd07815e9f0 legacy=0x7fd07806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd075400700 1 -- 172.21.15.157:0/510049951 >> v1:172.21.15.3:6789/0 conn(0x7fd078157480 legacy=0x7fd0780b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd075400700 1 -- 172.21.15.157:0/510049951 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07806d790 con 0x7fd0780b4b40 2024-09-13T16:50:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd075400700 1 --2- 172.21.15.157:0/510049951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd07815e4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd064009940 tx=0x7fd064009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bcb541a54d2eeb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:07.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd074a00700 1 -- 172.21.15.157:0/510049951 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400ba60 con 0x7fd0780b4b40 2024-09-13T16:50:07.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd074a00700 1 -- 172.21.15.157:0/510049951 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064009e90 con 0x7fd0780b4b40 2024-09-13T16:50:07.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd074a00700 1 -- 172.21.15.157:0/510049951 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400bde0 con 0x7fd0780b4b40 2024-09-13T16:50:07.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 -- 172.21.15.157:0/510049951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 msgr2=0x7fd07815e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 --2- 172.21.15.157:0/510049951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd07815e4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd064009940 tx=0x7fd064009c50 comp rx=0 tx=0).stop 2024-09-13T16:50:07.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 -- 172.21.15.157:0/510049951 shutdown_connections 2024-09-13T16:50:07.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 --2- 172.21.15.157:0/510049951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd07815e4b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 -- 172.21.15.157:0/510049951 >> 172.21.15.157:0/510049951 conn(0x7fd078071f40 msgr2=0x7fd07806b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:07.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 -- 172.21.15.157:0/510049951 shutdown_connections 2024-09-13T16:50:07.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.884+0000 7fd07ce00700 1 -- 172.21.15.157:0/510049951 wait complete. 2024-09-13T16:50:07.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.888+0000 7fd07ce00700 1 Processor -- start 2024-09-13T16:50:07.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.888+0000 7fd07ce00700 1 -- start start 2024-09-13T16:50:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.888+0000 7fd07ce00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd0780ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.888+0000 7fd07ce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078157480 0x7fd07815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.888+0000 7fd07ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd07815e9f0 0x7fd07814d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.888+0000 7fd07ce00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780b0580 con 0x7fd07815e9f0 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.888+0000 7fd07ce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07815ab30 con 0x7fd078157480 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.888+0000 7fd07ce00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0781f3830 con 0x7fd0780b4b40 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd0780ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078157480 0x7fd07815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd0780ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49460/0 (socket says 172.21.15.157:49460) 2024-09-13T16:50:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075e00700 1 -- 172.21.15.157:0/58591899 learned_addr learned my addr 172.21.15.157:0/58591899 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd076800700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd07815e9f0 0x7fd07814d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075400700 1 -- 172.21.15.157:0/58591899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 msgr2=0x7fd0780ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075400700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd0780ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075400700 1 -- 172.21.15.157:0/58591899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd07815e9f0 msgr2=0x7fd07814d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075400700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd07815e9f0 0x7fd07814d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075400700 1 -- 172.21.15.157:0/58591899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064009570 con 0x7fd078157480 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd076800700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd07815e9f0 0x7fd07814d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd075400700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078157480 0x7fd07815a5f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd064016000 tx=0x7fd064004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd06e000700 1 -- 172.21.15.157:0/58591899 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0640036c0 con 0x7fd078157480 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd06e000700 1 -- 172.21.15.157:0/58591899 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06401e070 con 0x7fd078157480 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd06e000700 1 -- 172.21.15.157:0/58591899 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064003aa0 con 0x7fd078157480 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0780ad8d0 con 0x7fd078157480 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0780afbe0 con 0x7fd078157480 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd06e000700 1 -- 172.21.15.157:0/58591899 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd064025480 con 0x7fd078157480 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd06e000700 1 -- 172.21.15.157:0/58591899 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fd064020070 con 0x7fd078157480 2024-09-13T16:50:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.892+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0780b06c0 con 0x7fd078157480 2024-09-13T16:50:07.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.896+0000 7fd06e000700 1 -- 172.21.15.157:0/58591899 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fd064044e30 con 0x7fd078157480 2024-09-13T16:50:07.952 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:07.948+0000 7f8ef30d62c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:50:07.973 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:07.969+0000 7fc3872eb2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:50:07.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.976+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "02190853-86bc-48c1-b718-2b150b722955", "id": 1} v 0) v1 -- 0x7fd07815e4b0 con 0x7fd078157480 2024-09-13T16:50:07.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.984+0000 7fd06e000700 1 -- 172.21.15.157:0/58591899 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "02190853-86bc-48c1-b718-2b150b722955", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd0640330b0 con 0x7fd078157480 2024-09-13T16:50:07.991 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078157480 msgr2=0x7fd07815a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078157480 0x7fd07815a5f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd064016000 tx=0x7fd064004770 comp rx=0 tx=0).stop 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 shutdown_connections 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd07815e9f0 0x7fd07814d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078157480 0x7fd07815a5f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 --2- 172.21.15.157:0/58591899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b4b40 0x7fd0780ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 >> 172.21.15.157:0/58591899 conn(0x7fd078071f40 msgr2=0x7fd0780b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 shutdown_connections 2024-09-13T16:50:07.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:07.988+0000 7fd07ce00700 1 -- 172.21.15.157:0/58591899 wait complete. 2024-09-13T16:50:08.013 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 321a1109-12e8-4ce4-b73a-9971e6a42760 2 2024-09-13T16:50:08.158 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:08.152+0000 7f8ef30d62c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:50:08.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.160+0000 7f3501c00700 1 Processor -- start 2024-09-13T16:50:08.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501c00700 1 -- start start 2024-09-13T16:50:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34fc06aaa0 0x7f34fc16bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc0ce520 con 0x7f34fc0b5b30 2024-09-13T16:50:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc0ce800 con 0x7f34fc160fc0 2024-09-13T16:50:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc0ceae0 con 0x7f34fc06aaa0 2024-09-13T16:50:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f34fbe00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f34fc160fc0 0x7f34fc0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47172/0 (socket says 172.21.15.157:47172) 2024-09-13T16:50:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f34fbe00700 1 -- 172.21.15.157:0/3450137844 learned_addr learned my addr 172.21.15.157:0/3450137844 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-09-13T16:50:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501200700 1 --2- 172.21.15.157:0/3450137844 >> v2:172.21.15.3:3300/0 conn(0x7f34fc06aaa0 0x7f34fc16bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f34faa00700 1 -- 172.21.15.157:0/3450137844 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510690483 0 0) 0x7f34fc0ce800 con 0x7f34fc160fc0 2024-09-13T16:50:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f34faa00700 1 -- 172.21.15.157:0/3450137844 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e00035d0 con 0x7f34fc160fc0 2024-09-13T16:50:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501200700 1 -- 172.21.15.157:0/3450137844 >> v1:172.21.15.157:6789/0 conn(0x7f34fc160fc0 legacy=0x7f34fc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f34faa00700 1 -- 172.21.15.157:0/3450137844 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3999642457 0 0) 0x7f34e00035d0 con 0x7f34fc160fc0 2024-09-13T16:50:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501200700 1 -- 172.21.15.157:0/3450137844 >> v1:172.21.15.3:6789/0 conn(0x7f34fc0b5b30 legacy=0x7f34fc06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.164+0000 7f3501200700 1 -- 172.21.15.157:0/3450137844 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc06c660 con 0x7f34fc06aaa0 2024-09-13T16:50:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f3501200700 1 --2- 172.21.15.157:0/3450137844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc06aaa0 0x7f34fc16bcb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009d90 tx=0x7f34f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04d2032edc9305e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f34faa00700 1 -- 172.21.15.157:0/3450137844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400e070 con 0x7f34fc06aaa0 2024-09-13T16:50:08.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f34faa00700 1 -- 172.21.15.157:0/3450137844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f4003a40 con 0x7f34fc06aaa0 2024-09-13T16:50:08.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f34faa00700 1 -- 172.21.15.157:0/3450137844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec001670 con 0x7f34fc06aaa0 2024-09-13T16:50:08.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f3501c00700 1 -- 172.21.15.157:0/3450137844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc06aaa0 msgr2=0x7f34fc16bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f3501c00700 1 --2- 172.21.15.157:0/3450137844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc06aaa0 0x7f34fc16bcb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009d90 tx=0x7f34f400b330 comp rx=0 tx=0).stop 2024-09-13T16:50:08.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f3501c00700 1 -- 172.21.15.157:0/3450137844 shutdown_connections 2024-09-13T16:50:08.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f3501c00700 1 --2- 172.21.15.157:0/3450137844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc06aaa0 0x7f34fc16bcb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f3501c00700 1 -- 172.21.15.157:0/3450137844 >> 172.21.15.157:0/3450137844 conn(0x7f34fc071da0 msgr2=0x7f34fc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:08.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f3501c00700 1 -- 172.21.15.157:0/3450137844 shutdown_connections 2024-09-13T16:50:08.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.168+0000 7f3501c00700 1 -- 172.21.15.157:0/3450137844 wait complete. 2024-09-13T16:50:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501c00700 1 Processor -- start 2024-09-13T16:50:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501c00700 1 -- start start 2024-09-13T16:50:08.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34fc06aaa0 0x7f34fc0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc0b5b30 0x7f34fc150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc160fc0 0x7f34fc15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0affa0 con 0x7f34fc0b5b30 2024-09-13T16:50:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0aef00 con 0x7f34fc160fc0 2024-09-13T16:50:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0af1e0 con 0x7f34fc06aaa0 2024-09-13T16:50:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34fc06aaa0 0x7f34fc0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34fc06aaa0 0x7f34fc0ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49472/0 (socket says 172.21.15.157:49472) 2024-09-13T16:50:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 -- 172.21.15.157:0/3649144569 learned_addr learned my addr 172.21.15.157:0/3649144569 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fb400700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc0b5b30 0x7f34fc150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f3501200700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc160fc0 0x7f34fc15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.180 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:08.173+0000 7fc3872eb2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 -- 172.21.15.157:0/3649144569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc160fc0 msgr2=0x7f34fc15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc160fc0 0x7f34fc15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 -- 172.21.15.157:0/3649144569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc0b5b30 msgr2=0x7f34fc150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc0b5b30 0x7f34fc150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 -- 172.21.15.157:0/3649144569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4009a70 con 0x7f34fc06aaa0 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34fbe00700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34fc06aaa0 0x7f34fc0ae620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00c520 tx=0x7f34ec00c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.172+0000 7f34f3e00700 1 -- 172.21.15.157:0/3649144569 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec004ca0 con 0x7f34fc06aaa0 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.176+0000 7f34f3e00700 1 -- 172.21.15.157:0/3649144569 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec004e20 con 0x7f34fc06aaa0 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.176+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc0af4c0 con 0x7f34fc06aaa0 2024-09-13T16:50:08.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.176+0000 7f34f3e00700 1 -- 172.21.15.157:0/3649144569 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec007310 con 0x7f34fc06aaa0 2024-09-13T16:50:08.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.176+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc0afa80 con 0x7f34fc06aaa0 2024-09-13T16:50:08.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.176+0000 7f34f3e00700 1 -- 172.21.15.157:0/3649144569 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f34ec01d3f0 con 0x7f34fc06aaa0 2024-09-13T16:50:08.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.176+0000 7f34f3e00700 1 -- 172.21.15.157:0/3649144569 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f34ec0137e0 con 0x7f34fc06aaa0 2024-09-13T16:50:08.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.176+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc0afca0 con 0x7f34fc06aaa0 2024-09-13T16:50:08.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.180+0000 7f34f3e00700 1 -- 172.21.15.157:0/3649144569 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f34ec013470 con 0x7f34fc06aaa0 2024-09-13T16:50:08.266 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.260+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "321a1109-12e8-4ce4-b73a-9971e6a42760", "id": 2} v 0) v1 -- 0x7f34fc06a390 con 0x7f34fc06aaa0 2024-09-13T16:50:08.276 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f34f3e00700 1 -- 172.21.15.157:0/3649144569 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "321a1109-12e8-4ce4-b73a-9971e6a42760", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f34ec013a30 con 0x7f34fc06aaa0 2024-09-13T16:50:08.276 INFO:teuthology.orchestra.run.smithi157.stdout:2 2024-09-13T16:50:08.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34fc06aaa0 msgr2=0x7f34fc0ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34fc06aaa0 0x7f34fc0ae620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00c520 tx=0x7f34ec00c8e0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 shutdown_connections 2024-09-13T16:50:08.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc160fc0 0x7f34fc15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc0b5b30 0x7f34fc150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 --2- 172.21.15.157:0/3649144569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34fc06aaa0 0x7f34fc0ae620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 >> 172.21.15.157:0/3649144569 conn(0x7f34fc071da0 msgr2=0x7f34fc0cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:08.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 shutdown_connections 2024-09-13T16:50:08.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.272+0000 7f3501c00700 1 -- 172.21.15.157:0/3649144569 wait complete. 2024-09-13T16:50:08.300 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 35421139-a6b7-48fb-a6cd-4521d2746666 3 2024-09-13T16:50:08.442 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:08.436+0000 7f8ef30d62c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:50:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.444+0000 7f511de00700 1 Processor -- start 2024-09-13T16:50:08.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f511de00700 1 -- start start 2024-09-13T16:50:08.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f511de00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f511815e7c0 0x7f511815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f511de00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511815b7c0 con 0x7f51180b5250 2024-09-13T16:50:08.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f511de00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51180ce1e0 con 0x7f5118155250 2024-09-13T16:50:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f511de00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511815b1d0 con 0x7f511815e7c0 2024-09-13T16:50:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f510fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f511815e7c0 0x7f511815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f510fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f511815e7c0 0x7f511815ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52678/0 (socket says 172.21.15.157:52678) 2024-09-13T16:50:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f510fe00700 1 -- 172.21.15.157:0/3817030462 learned_addr learned my addr 172.21.15.157:0/3817030462 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f510fe00700 1 -- 172.21.15.157:0/3817030462 >> v1:172.21.15.157:6789/0 conn(0x7f5118155250 legacy=0x7f511815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f510fe00700 1 -- 172.21.15.157:0/3817030462 >> v1:172.21.15.3:6789/0 conn(0x7f51180b5250 legacy=0x7f511806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f510fe00700 1 -- 172.21.15.157:0/3817030462 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511815f280 con 0x7f511815e7c0 2024-09-13T16:50:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f510fe00700 1 --2- 172.21.15.157:0/3817030462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 0x7f511815ec20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5100009910 tx=0x7f5100009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ffd878d5ba8f6a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f5116000700 1 -- 172.21.15.157:0/3817030462 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000ba60 con 0x7f511815e7c0 2024-09-13T16:50:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f5116000700 1 -- 172.21.15.157:0/3817030462 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510000bbc0 con 0x7f511815e7c0 2024-09-13T16:50:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.448+0000 7f5116000700 1 -- 172.21.15.157:0/3817030462 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100009e60 con 0x7f511815e7c0 2024-09-13T16:50:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 -- 172.21.15.157:0/3817030462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 msgr2=0x7f511815ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 --2- 172.21.15.157:0/3817030462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 0x7f511815ec20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5100009910 tx=0x7f5100009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 -- 172.21.15.157:0/3817030462 shutdown_connections 2024-09-13T16:50:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 --2- 172.21.15.157:0/3817030462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 0x7f511815ec20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 -- 172.21.15.157:0/3817030462 >> 172.21.15.157:0/3817030462 conn(0x7f5118071f10 msgr2=0x7f511806b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 -- 172.21.15.157:0/3817030462 shutdown_connections 2024-09-13T16:50:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 -- 172.21.15.157:0/3817030462 wait complete. 2024-09-13T16:50:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 Processor -- start 2024-09-13T16:50:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.452+0000 7f511de00700 1 -- start start 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f511de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180b5250 0x7f511814d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f511de00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5118155250 0x7f5118157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f511de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 0x7f511815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f511de00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511814f110 con 0x7f51180b5250 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f511de00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511814e500 con 0x7f511815e7c0 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f511de00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511814e7e0 con 0x7f5118155250 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5118155250 0x7f5118157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5118155250 0x7f5118157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49488/0 (socket says 172.21.15.157:49488) 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 -- 172.21.15.157:0/1575580201 learned_addr learned my addr 172.21.15.157:0/1575580201 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f5117400700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 0x7f511815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f5116a00700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180b5250 0x7f511814d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 -- 172.21.15.157:0/1575580201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 msgr2=0x7f511815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 0x7f511815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 -- 172.21.15.157:0/1575580201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180b5250 msgr2=0x7f511814d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180b5250 0x7f511814d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 -- 172.21.15.157:0/1575580201 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100009570 con 0x7f5118155250 2024-09-13T16:50:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510fe00700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5118155250 0x7f5118157e80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5100014040 tx=0x7f5100004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510f400700 1 -- 172.21.15.157:0/1575580201 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100004610 con 0x7f5118155250 2024-09-13T16:50:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510f400700 1 -- 172.21.15.157:0/1575580201 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510001f070 con 0x7f5118155250 2024-09-13T16:50:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511814eac0 con 0x7f5118155250 2024-09-13T16:50:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510f400700 1 -- 172.21.15.157:0/1575580201 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000038e0 con 0x7f5118155250 2024-09-13T16:50:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511814de90 con 0x7f5118155250 2024-09-13T16:50:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510f400700 1 -- 172.21.15.157:0/1575580201 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5100003d40 con 0x7f5118155250 2024-09-13T16:50:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.456+0000 7f510f400700 1 -- 172.21.15.157:0/1575580201 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f5100021070 con 0x7f5118155250 2024-09-13T16:50:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.460+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511814f280 con 0x7f5118155250 2024-09-13T16:50:08.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.460+0000 7f510f400700 1 -- 172.21.15.157:0/1575580201 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f510002b020 con 0x7f5118155250 2024-09-13T16:50:08.472 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:08.469+0000 7fc3872eb2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:50:08.523 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:08.516+0000 7f8ef30d62c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:50:08.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.540+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "35421139-a6b7-48fb-a6cd-4521d2746666", "id": 3} v 0) v1 -- 0x7f511815ec20 con 0x7f5118155250 2024-09-13T16:50:08.550 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:08.545+0000 7fc3872eb2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:50:08.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.548+0000 7f510f400700 1 -- 172.21.15.157:0/1575580201 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35421139-a6b7-48fb-a6cd-4521d2746666", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5100015020 con 0x7f5118155250 2024-09-13T16:50:08.555 INFO:teuthology.orchestra.run.smithi157.stdout:3 2024-09-13T16:50:08.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5118155250 msgr2=0x7f5118157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5118155250 0x7f5118157e80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5100014040 tx=0x7f5100004770 comp rx=0 tx=0).stop 2024-09-13T16:50:08.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 shutdown_connections 2024-09-13T16:50:08.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511815e7c0 0x7f511815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5118155250 0x7f5118157e80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 --2- 172.21.15.157:0/1575580201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180b5250 0x7f511814d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 >> 172.21.15.157:0/1575580201 conn(0x7f5118071f10 msgr2=0x7f511815def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:08.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 shutdown_connections 2024-09-13T16:50:08.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.552+0000 7f511de00700 1 -- 172.21.15.157:0/1575580201 wait complete. 2024-09-13T16:50:08.577 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new a4cb098f-d503-43f5-849a-ed8049e85047 4 2024-09-13T16:50:08.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac9200700 1 Processor -- start 2024-09-13T16:50:08.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac9200700 1 -- start start 2024-09-13T16:50:08.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac9200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ac40cda60 0x7f4ac40af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac9200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac406ca90 0x7f4ac4071960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac9200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac40cfce0 con 0x7f4ac40cde40 2024-09-13T16:50:08.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac9200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac415c030 con 0x7f4ac40cda60 2024-09-13T16:50:08.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac9200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40cf720 con 0x7f4ac406ca90 2024-09-13T16:50:08.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac2800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac406ca90 0x7f4ac4071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ab9400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4ac40cde40 0x7f4ac415b920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47194/0 (socket says 172.21.15.157:47194) 2024-09-13T16:50:08.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ab9400700 1 -- 172.21.15.157:0/827062951 learned_addr learned my addr 172.21.15.157:0/827062951 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-09-13T16:50:08.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac1e00700 1 --2- 172.21.15.157:0/827062951 >> v2:172.21.15.3:3300/0 conn(0x7f4ac40cda60 0x7f4ac40af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac2800700 1 -- 172.21.15.157:0/827062951 >> v2:172.21.15.3:3300/0 conn(0x7f4ac40cda60 msgr2=0x7f4ac40af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:08.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac1400700 1 -- 172.21.15.157:0/827062951 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380948836 0 0) 0x7f4ac40cfce0 con 0x7f4ac40cde40 2024-09-13T16:50:08.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac2800700 1 --2- 172.21.15.157:0/827062951 >> v2:172.21.15.3:3300/0 conn(0x7f4ac40cda60 0x7f4ac40af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac2800700 1 -- 172.21.15.157:0/827062951 >> v1:172.21.15.157:6789/0 conn(0x7f4ac40cde40 legacy=0x7f4ac415b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac2800700 1 -- 172.21.15.157:0/827062951 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40cfa00 con 0x7f4ac406ca90 2024-09-13T16:50:08.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac2800700 1 --2- 172.21.15.157:0/827062951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac406ca90 0x7f4ac4071960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54e27835ecfbbf47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:08.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.736+0000 7f4ac1400700 1 -- 172.21.15.157:0/827062951 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00e070 con 0x7f4ac406ca90 2024-09-13T16:50:08.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac1400700 1 -- 172.21.15.157:0/827062951 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc00bbd0 con 0x7f4ac406ca90 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac1400700 1 -- 172.21.15.157:0/827062951 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc012430 con 0x7f4ac406ca90 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac9200700 1 -- 172.21.15.157:0/827062951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac406ca90 msgr2=0x7f4ac4071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac9200700 1 --2- 172.21.15.157:0/827062951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac406ca90 0x7f4ac4071960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc00b330 comp rx=0 tx=0).stop 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac9200700 1 -- 172.21.15.157:0/827062951 shutdown_connections 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac9200700 1 --2- 172.21.15.157:0/827062951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac406ca90 0x7f4ac4071960 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac9200700 1 --2- 172.21.15.157:0/827062951 >> v2:172.21.15.3:3300/0 conn(0x7f4ac40cda60 0x7f4ac40af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac9200700 1 -- 172.21.15.157:0/827062951 >> 172.21.15.157:0/827062951 conn(0x7f4ac406dff0 msgr2=0x7f4ac4071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac9200700 1 -- 172.21.15.157:0/827062951 shutdown_connections 2024-09-13T16:50:08.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.740+0000 7f4ac9200700 1 -- 172.21.15.157:0/827062951 wait complete. 2024-09-13T16:50:08.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac9200700 1 Processor -- start 2024-09-13T16:50:08.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac9200700 1 -- start start 2024-09-13T16:50:08.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac9200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ac406ca90 0x7f4ac414ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac9200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac40cda60 0x7f4ac406a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac40cde40 0x7f4ac4157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:08.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac9200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac414f4b0 con 0x7f4ac406ca90 2024-09-13T16:50:08.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac9200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac414ef80 con 0x7f4ac40cde40 2024-09-13T16:50:08.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac9200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac41574b0 con 0x7f4ac40cda60 2024-09-13T16:50:08.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac40cda60 0x7f4ac406a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac40cda60 0x7f4ac406a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49498/0 (socket says 172.21.15.157:49498) 2024-09-13T16:50:08.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 -- 172.21.15.157:0/21210178 learned_addr learned my addr 172.21.15.157:0/21210178 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:08.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac2800700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac40cde40 0x7f4ac4157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ac1e00700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ac406ca90 0x7f4ac414ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 -- 172.21.15.157:0/21210178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac40cde40 msgr2=0x7f4ac4157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac40cde40 0x7f4ac4157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 -- 172.21.15.157:0/21210178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ac406ca90 msgr2=0x7f4ac414ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ac406ca90 0x7f4ac414ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 -- 172.21.15.157:0/21210178 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc009a70 con 0x7f4ac40cda60 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.744+0000 7f4ab9400700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac40cda60 0x7f4ac406a390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00c510 tx=0x7f4aac00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.748+0000 7f4abaa00700 1 -- 172.21.15.157:0/21210178 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac004ca0 con 0x7f4ac40cda60 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.748+0000 7f4abaa00700 1 -- 172.21.15.157:0/21210178 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac004e20 con 0x7f4ac40cda60 2024-09-13T16:50:08.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.748+0000 7f4abaa00700 1 -- 172.21.15.157:0/21210178 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac007310 con 0x7f4ac40cda60 2024-09-13T16:50:08.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.748+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac41514f0 con 0x7f4ac40cda60 2024-09-13T16:50:08.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.748+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac4150ea0 con 0x7f4ac40cda60 2024-09-13T16:50:08.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.748+0000 7f4abaa00700 1 -- 172.21.15.157:0/21210178 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4aac0133c0 con 0x7f4ac40cda60 2024-09-13T16:50:08.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.748+0000 7f4abaa00700 1 -- 172.21.15.157:0/21210178 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f4aac013b80 con 0x7f4ac40cda60 2024-09-13T16:50:08.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.748+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac4158a70 con 0x7f4ac40cda60 2024-09-13T16:50:08.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.752+0000 7f4abaa00700 1 -- 172.21.15.157:0/21210178 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f4aac013d40 con 0x7f4ac40cda60 2024-09-13T16:50:08.773 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:08.768+0000 7f8ef30d62c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:50:08.802 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:08.797+0000 7fc3872eb2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:50:08.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.828+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a4cb098f-d503-43f5-849a-ed8049e85047", "id": 4} v 0) v1 -- 0x7f4ac4071960 con 0x7f4ac40cda60 2024-09-13T16:50:08.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4abaa00700 1 -- 172.21.15.157:0/21210178 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4cb098f-d503-43f5-849a-ed8049e85047", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4aac013670 con 0x7f4ac40cda60 2024-09-13T16:50:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:4 2024-09-13T16:50:08.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac40cda60 msgr2=0x7f4ac406a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:08.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac40cda60 0x7f4ac406a390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00c510 tx=0x7f4aac00c820 comp rx=0 tx=0).stop 2024-09-13T16:50:08.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 shutdown_connections 2024-09-13T16:50:08.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac40cde40 0x7f4ac4157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac40cda60 0x7f4ac406a390 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 --2- 172.21.15.157:0/21210178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ac406ca90 0x7f4ac414ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:08.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 >> 172.21.15.157:0/21210178 conn(0x7f4ac406dff0 msgr2=0x7f4ac4158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:08.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 shutdown_connections 2024-09-13T16:50:08.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:08.840+0000 7f4ac9200700 1 -- 172.21.15.157:0/21210178 wait complete. 2024-09-13T16:50:08.865 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:08.860+0000 7f8ef30d62c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:50:08.869 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 8df79f9b-285f-4f73-aac8-9b98c1554a71 5 2024-09-13T16:50:08.888 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:08.881+0000 7fc3872eb2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:50:08.970 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:08.964+0000 7f8ef30d62c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:50:08.990 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:08.985+0000 7fc3872eb2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:50:09.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67be00700 1 Processor -- start 2024-09-13T16:50:09.020 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67be00700 1 -- start start 2024-09-13T16:50:09.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b4510 0x7fc67c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc67c0b48f0 0x7fc67c0cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67be00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc67c0cd5b0 0x7fc67c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.021 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67be00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67c0cc670 con 0x7fc67c0b48f0 2024-09-13T16:50:09.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67be00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67c15e1d0 con 0x7fc67c0cd5b0 2024-09-13T16:50:09.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67be00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c0b56a0 con 0x7fc67c0b4510 2024-09-13T16:50:09.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b4510 0x7fc67c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b4510 0x7fc67c15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49508/0 (socket says 172.21.15.157:49508) 2024-09-13T16:50:09.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 -- 172.21.15.157:0/1858745345 learned_addr learned my addr 172.21.15.157:0/1858745345 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 -- 172.21.15.157:0/1858745345 >> v2:172.21.15.157:3300/0 conn(0x7fc67c0cd5b0 msgr2=0x7fc67c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc672000700 1 --2- 172.21.15.157:0/1858745345 >> v2:172.21.15.3:3300/0 conn(0x7fc67c0b48f0 0x7fc67c0cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67b400700 1 --2- 172.21.15.157:0/1858745345 >> v2:172.21.15.157:3300/0 conn(0x7fc67c0cd5b0 0x7fc67c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 --2- 172.21.15.157:0/1858745345 >> v2:172.21.15.157:3300/0 conn(0x7fc67c0cd5b0 0x7fc67c0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 -- 172.21.15.157:0/1858745345 >> v2:172.21.15.3:3300/0 conn(0x7fc67c0b48f0 msgr2=0x7fc67c0cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 --2- 172.21.15.157:0/1858745345 >> v2:172.21.15.3:3300/0 conn(0x7fc67c0b48f0 0x7fc67c0cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 -- 172.21.15.157:0/1858745345 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c0b5980 con 0x7fc67c0b4510 2024-09-13T16:50:09.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc672000700 1 --2- 172.21.15.157:0/1858745345 >> v2:172.21.15.3:3300/0 conn(0x7fc67c0b48f0 0x7fc67c0cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:09.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67b400700 1 --2- 172.21.15.157:0/1858745345 >> v2:172.21.15.157:3300/0 conn(0x7fc67c0cd5b0 0x7fc67c0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:09.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67aa00700 1 --2- 172.21.15.157:0/1858745345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b4510 0x7fc67c15dc90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0b5140 tx=0x7fc66c009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbca59dc747d30c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67a000700 1 -- 172.21.15.157:0/1858745345 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00b8a0 con 0x7fc67c0b4510 2024-09-13T16:50:09.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67a000700 1 -- 172.21.15.157:0/1858745345 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c00ba00 con 0x7fc67c0b4510 2024-09-13T16:50:09.024 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.016+0000 7fc67a000700 1 -- 172.21.15.157:0/1858745345 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00bd90 con 0x7fc67c0b4510 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- 172.21.15.157:0/1858745345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b4510 msgr2=0x7fc67c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 --2- 172.21.15.157:0/1858745345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b4510 0x7fc67c15dc90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0b5140 tx=0x7fc66c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- 172.21.15.157:0/1858745345 shutdown_connections 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 --2- 172.21.15.157:0/1858745345 >> v2:172.21.15.3:3300/0 conn(0x7fc67c0b48f0 0x7fc67c0cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 --2- 172.21.15.157:0/1858745345 >> v2:172.21.15.157:3300/0 conn(0x7fc67c0cd5b0 0x7fc67c0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 --2- 172.21.15.157:0/1858745345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b4510 0x7fc67c15dc90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- 172.21.15.157:0/1858745345 >> 172.21.15.157:0/1858745345 conn(0x7fc67c071da0 msgr2=0x7fc67c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- 172.21.15.157:0/1858745345 shutdown_connections 2024-09-13T16:50:09.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- 172.21.15.157:0/1858745345 wait complete. 2024-09-13T16:50:09.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 Processor -- start 2024-09-13T16:50:09.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- start start 2024-09-13T16:50:09.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc67c0b4510 0x7fc67c14d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b48f0 0x7fc67c14f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc67c0cd5b0 0x7fc67c14d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c14d780 con 0x7fc67c0cd5b0 2024-09-13T16:50:09.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c1f3250 con 0x7fc67c0b4510 2024-09-13T16:50:09.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67be00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c1f3530 con 0x7fc67c0b48f0 2024-09-13T16:50:09.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.020+0000 7fc67aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc67c0b4510 0x7fc67c14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc67c0b4510 0x7fc67c14d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52736/0 (socket says 172.21.15.157:52736) 2024-09-13T16:50:09.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67aa00700 1 -- 172.21.15.157:0/1618367867 learned_addr learned my addr 172.21.15.157:0/1618367867 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:09.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc672000700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b48f0 0x7fc67c14f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67b400700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc67c0cd5b0 0x7fc67c14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67aa00700 1 -- 172.21.15.157:0/1618367867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b48f0 msgr2=0x7fc67c14f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67aa00700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b48f0 0x7fc67c14f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67aa00700 1 -- 172.21.15.157:0/1618367867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc67c0cd5b0 msgr2=0x7fc67c14d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67aa00700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc67c0cd5b0 0x7fc67c14d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67aa00700 1 -- 172.21.15.157:0/1618367867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c009570 con 0x7fc67c0b4510 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67aa00700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc67c0b4510 0x7fc67c14d240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc66c014040 tx=0x7fc66c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc673400700 1 -- 172.21.15.157:0/1618367867 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0038c0 con 0x7fc67c0b4510 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc673400700 1 -- 172.21.15.157:0/1618367867 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c01e070 con 0x7fc67c0b4510 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc673400700 1 -- 172.21.15.157:0/1618367867 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c003c40 con 0x7fc67c0b4510 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67c14ee10 con 0x7fc67c0b4510 2024-09-13T16:50:09.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67c14f3d0 con 0x7fc67c0b4510 2024-09-13T16:50:09.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc673400700 1 -- 172.21.15.157:0/1618367867 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fc66c020030 con 0x7fc67c0b4510 2024-09-13T16:50:09.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc673400700 1 -- 172.21.15.157:0/1618367867 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fc66c003da0 con 0x7fc67c0b4510 2024-09-13T16:50:09.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.024+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67c06edc0 con 0x7fc67c0b4510 2024-09-13T16:50:09.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.028+0000 7fc673400700 1 -- 172.21.15.157:0/1618367867 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fc66c015020 con 0x7fc67c0b4510 2024-09-13T16:50:09.109 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.104+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8df79f9b-285f-4f73-aac8-9b98c1554a71", "id": 5} v 0) v1 -- 0x7fc67c0a85f0 con 0x7fc67c0b4510 2024-09-13T16:50:09.119 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.112+0000 7fc673400700 1 -- 172.21.15.157:0/1618367867 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8df79f9b-285f-4f73-aac8-9b98c1554a71", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc66c02c850 con 0x7fc67c0b4510 2024-09-13T16:50:09.119 INFO:teuthology.orchestra.run.smithi157.stdout:5 2024-09-13T16:50:09.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc67c0b4510 msgr2=0x7fc67c14d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc67c0b4510 0x7fc67c14d240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc66c014040 tx=0x7fc66c004770 comp rx=0 tx=0).stop 2024-09-13T16:50:09.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 shutdown_connections 2024-09-13T16:50:09.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc67c0cd5b0 0x7fc67c14d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67c0b48f0 0x7fc67c14f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 --2- 172.21.15.157:0/1618367867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc67c0b4510 0x7fc67c14d240 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 >> 172.21.15.157:0/1618367867 conn(0x7fc67c071da0 msgr2=0x7fc67c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:09.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 shutdown_connections 2024-09-13T16:50:09.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.116+0000 7fc67be00700 1 -- 172.21.15.157:0/1618367867 wait complete. 2024-09-13T16:50:09.142 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01 6 2024-09-13T16:50:09.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.296+0000 7fef73000700 1 Processor -- start 2024-09-13T16:50:09.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.300+0000 7fef73000700 1 -- start start 2024-09-13T16:50:09.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.300+0000 7fef73000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c153d80 0x7fef6c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.300+0000 7fef73000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef6c15abf0 0x7fef6c0b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.300+0000 7fef73000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fef6c154160 0x7fef6c15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c15e450 con 0x7fef6c154160 2024-09-13T16:50:09.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c159f00 con 0x7fef6c15abf0 2024-09-13T16:50:09.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c15e170 con 0x7fef6c153d80 2024-09-13T16:50:09.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef6c15abf0 0x7fef6c0b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c153d80 0x7fef6c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef6c15abf0 0x7fef6c0b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52750/0 (socket says 172.21.15.157:52750) 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 -- 172.21.15.157:0/779222644 learned_addr learned my addr 172.21.15.157:0/779222644 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef72600700 1 --2- 172.21.15.157:0/779222644 >> v2:172.21.15.3:3300/0 conn(0x7fef6c154160 0x7fef6c15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 -- 172.21.15.157:0/779222644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c153d80 msgr2=0x7fef6c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 --2- 172.21.15.157:0/779222644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c153d80 0x7fef6c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 -- 172.21.15.157:0/779222644 >> v2:172.21.15.3:3300/0 conn(0x7fef6c154160 msgr2=0x7fef6c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 --2- 172.21.15.157:0/779222644 >> v2:172.21.15.3:3300/0 conn(0x7fef6c154160 0x7fef6c15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 -- 172.21.15.157:0/779222644 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c15a110 con 0x7fef6c15abf0 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef71200700 1 --2- 172.21.15.157:0/779222644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c15abf0 0x7fef6c0b4340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0b2ab0 tx=0x7fef58009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5151d5d2624e169 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef6be00700 1 -- 172.21.15.157:0/779222644 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58011830 con 0x7fef6c15abf0 2024-09-13T16:50:09.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef6be00700 1 -- 172.21.15.157:0/779222644 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef58011990 con 0x7fef6c15abf0 2024-09-13T16:50:09.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef6be00700 1 -- 172.21.15.157:0/779222644 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58011d20 con 0x7fef6c15abf0 2024-09-13T16:50:09.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 -- 172.21.15.157:0/779222644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c15abf0 msgr2=0x7fef6c0b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 --2- 172.21.15.157:0/779222644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c15abf0 0x7fef6c0b4340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0b2ab0 tx=0x7fef58009bf0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 -- 172.21.15.157:0/779222644 shutdown_connections 2024-09-13T16:50:09.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 --2- 172.21.15.157:0/779222644 >> v2:172.21.15.3:3300/0 conn(0x7fef6c154160 0x7fef6c15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 --2- 172.21.15.157:0/779222644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c15abf0 0x7fef6c0b4340 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 --2- 172.21.15.157:0/779222644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c153d80 0x7fef6c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 -- 172.21.15.157:0/779222644 >> 172.21.15.157:0/779222644 conn(0x7fef6c0712c0 msgr2=0x7fef6c0716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:09.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 -- 172.21.15.157:0/779222644 shutdown_connections 2024-09-13T16:50:09.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.304+0000 7fef73000700 1 -- 172.21.15.157:0/779222644 wait complete. 2024-09-13T16:50:09.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.308+0000 7fef73000700 1 Processor -- start 2024-09-13T16:50:09.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.308+0000 7fef73000700 1 -- start start 2024-09-13T16:50:09.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c153d80 0x7fef6c134930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.316 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef6c154160 0x7fef6c137be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c15abf0 0x7fef6c14ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c146b60 con 0x7fef6c154160 2024-09-13T16:50:09.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c135170 con 0x7fef6c153d80 2024-09-13T16:50:09.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c14a360 con 0x7fef6c15abf0 2024-09-13T16:50:09.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c153d80 0x7fef6c134930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c153d80 0x7fef6c134930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52752/0 (socket says 172.21.15.157:52752) 2024-09-13T16:50:09.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 -- 172.21.15.157:0/340196823 learned_addr learned my addr 172.21.15.157:0/340196823 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:09.317 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef72600700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c15abf0 0x7fef6c14ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71200700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef6c154160 0x7fef6c137be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 -- 172.21.15.157:0/340196823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c15abf0 msgr2=0x7fef6c14ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c15abf0 0x7fef6c14ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 -- 172.21.15.157:0/340196823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef6c154160 msgr2=0x7fef6c137be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef6c154160 0x7fef6c137be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 -- 172.21.15.157:0/340196823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58009570 con 0x7fef6c153d80 2024-09-13T16:50:09.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71200700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef6c154160 0x7fef6c137be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef71c00700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c153d80 0x7fef6c134930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fef6000e940 tx=0x7fef6000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef69600700 1 -- 172.21.15.157:0/340196823 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000ee90 con 0x7fef6c153d80 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef69600700 1 -- 172.21.15.157:0/340196823 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6000c850 con 0x7fef6c153d80 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c134e70 con 0x7fef6c153d80 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef69600700 1 -- 172.21.15.157:0/340196823 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000cbe0 con 0x7fef6c153d80 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c14a600 con 0x7fef6c153d80 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef69600700 1 -- 172.21.15.157:0/340196823 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fef60013770 con 0x7fef6c153d80 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef69600700 1 -- 172.21.15.157:0/340196823 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fef6000cd40 con 0x7fef6c153d80 2024-09-13T16:50:09.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.312+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c0cd400 con 0x7fef6c153d80 2024-09-13T16:50:09.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.316+0000 7fef69600700 1 -- 172.21.15.157:0/340196823 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fef6003be30 con 0x7fef6c153d80 2024-09-13T16:50:09.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.392+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01", "id": 6} v 0) v1 -- 0x7fef6c06ed70 con 0x7fef6c153d80 2024-09-13T16:50:09.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.404+0000 7fef69600700 1 -- 172.21.15.157:0/340196823 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef6002a020 con 0x7fef6c153d80 2024-09-13T16:50:09.410 INFO:teuthology.orchestra.run.smithi157.stdout:6 2024-09-13T16:50:09.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.404+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c153d80 msgr2=0x7fef6c134930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.404+0000 7fef73000700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c153d80 0x7fef6c134930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fef6000e940 tx=0x7fef6000ec50 comp rx=0 tx=0).stop 2024-09-13T16:50:09.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.404+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 shutdown_connections 2024-09-13T16:50:09.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.404+0000 7fef73000700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c15abf0 0x7fef6c14ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.404+0000 7fef73000700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef6c154160 0x7fef6c137be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.404+0000 7fef73000700 1 --2- 172.21.15.157:0/340196823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c153d80 0x7fef6c134930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.404+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 >> 172.21.15.157:0/340196823 conn(0x7fef6c0712c0 msgr2=0x7fef6c0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:09.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.408+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 shutdown_connections 2024-09-13T16:50:09.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.408+0000 7fef73000700 1 -- 172.21.15.157:0/340196823 wait complete. 2024-09-13T16:50:09.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new edeb379b-9a58-440e-911d-fd3119dd838d 7 2024-09-13T16:50:09.445 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:09.440+0000 7f8ef30d62c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:50:09.475 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:09.469+0000 7fc3872eb2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:50:09.592 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa05d200700 1 Processor -- start 2024-09-13T16:50:09.592 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa05d200700 1 -- start start 2024-09-13T16:50:09.593 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa05d200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0580b46b0 0x7fa05806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.593 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa05d200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.593 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa05d200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05815e400 con 0x7fa0580d04c0 2024-09-13T16:50:09.593 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa05d200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa058160670 con 0x7fa0580b46b0 2024-09-13T16:50:09.593 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa05d200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b57d0 con 0x7fa05806c960 2024-09-13T16:50:09.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa057400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa0580d04c0 0x7fa05815dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.157:49190/0 (socket says 172.21.15.157:49190) 2024-09-13T16:50:09.596 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa057400700 1 -- 172.21.15.157:0/2487221430 learned_addr learned my addr 172.21.15.157:0/2487221430 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-09-13T16:50:09.596 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa057e00700 1 --2- 172.21.15.157:0/2487221430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.596 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa056a00700 1 --2- 172.21.15.157:0/2487221430 >> v2:172.21.15.3:3300/0 conn(0x7fa0580b46b0 0x7fa05806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa056000700 1 -- 172.21.15.157:0/2487221430 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346623060 0 0) 0x7fa05815e400 con 0x7fa0580d04c0 2024-09-13T16:50:09.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa056000700 1 -- 172.21.15.157:0/2487221430 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa03c0035d0 con 0x7fa0580d04c0 2024-09-13T16:50:09.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa057e00700 1 -- 172.21.15.157:0/2487221430 >> v2:172.21.15.3:3300/0 conn(0x7fa0580b46b0 msgr2=0x7fa05806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:09.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa057e00700 1 --2- 172.21.15.157:0/2487221430 >> v2:172.21.15.3:3300/0 conn(0x7fa0580b46b0 0x7fa05806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa057e00700 1 -- 172.21.15.157:0/2487221430 >> v1:172.21.15.3:6789/0 conn(0x7fa0580d04c0 legacy=0x7fa05815dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.588+0000 7fa057e00700 1 -- 172.21.15.157:0/2487221430 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580b4c50 con 0x7fa05806c960 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa057e00700 1 --2- 172.21.15.157:0/2487221430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815e870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa050009d90 tx=0x7fa05000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd49e8d8c9c037e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa056000700 1 -- 172.21.15.157:0/2487221430 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000e070 con 0x7fa05806c960 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa056000700 1 -- 172.21.15.157:0/2487221430 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05000bb20 con 0x7fa05806c960 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa056000700 1 -- 172.21.15.157:0/2487221430 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050012430 con 0x7fa05806c960 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa05d200700 1 -- 172.21.15.157:0/2487221430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 msgr2=0x7fa05815e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa05d200700 1 --2- 172.21.15.157:0/2487221430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815e870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa050009d90 tx=0x7fa05000b330 comp rx=0 tx=0).stop 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa05d200700 1 -- 172.21.15.157:0/2487221430 shutdown_connections 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa05d200700 1 --2- 172.21.15.157:0/2487221430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815e870 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa05d200700 1 --2- 172.21.15.157:0/2487221430 >> v2:172.21.15.3:3300/0 conn(0x7fa0580b46b0 0x7fa05806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa05d200700 1 -- 172.21.15.157:0/2487221430 >> 172.21.15.157:0/2487221430 conn(0x7fa0580b2910 msgr2=0x7fa0580b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:09.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa05d200700 1 -- 172.21.15.157:0/2487221430 shutdown_connections 2024-09-13T16:50:09.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.592+0000 7fa05d200700 1 -- 172.21.15.157:0/2487221430 wait complete. 2024-09-13T16:50:09.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa05d200700 1 Processor -- start 2024-09-13T16:50:09.600 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa05d200700 1 -- start start 2024-09-13T16:50:09.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa05d200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa05d200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580b46b0 0x7fa0581f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa05d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0580d04c0 0x7fa0580cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa05d200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058157850 con 0x7fa0580b46b0 2024-09-13T16:50:09.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa05d200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058157b30 con 0x7fa0580d04c0 2024-09-13T16:50:09.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa05d200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa058157e10 con 0x7fa05806c960 2024-09-13T16:50:09.603 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa057400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa057e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0580d04c0 0x7fa0580cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa057e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0580d04c0 0x7fa0580cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52758/0 (socket says 172.21.15.157:52758) 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa057400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49568/0 (socket says 172.21.15.157:49568) 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa057e00700 1 -- 172.21.15.157:0/3690934524 learned_addr learned my addr 172.21.15.157:0/3690934524 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.596+0000 7fa056a00700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580b46b0 0x7fa0581f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa057e00700 1 -- 172.21.15.157:0/3690934524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 msgr2=0x7fa05815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa057e00700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa057e00700 1 -- 172.21.15.157:0/3690934524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580b46b0 msgr2=0x7fa0581f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa057e00700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580b46b0 0x7fa0581f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa057e00700 1 -- 172.21.15.157:0/3690934524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa050009a70 con 0x7fa0580d04c0 2024-09-13T16:50:09.604 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa057400700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:09.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa057e00700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0580d04c0 0x7fa0580cbf70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b680 tx=0x7fa050004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa04f400700 1 -- 172.21.15.157:0/3690934524 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050012430 con 0x7fa0580d04c0 2024-09-13T16:50:09.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa04f400700 1 -- 172.21.15.157:0/3690934524 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa050020070 con 0x7fa0580d04c0 2024-09-13T16:50:09.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0581580f0 con 0x7fa0580d04c0 2024-09-13T16:50:09.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa04f400700 1 -- 172.21.15.157:0/3690934524 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050004280 con 0x7fa0580d04c0 2024-09-13T16:50:09.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05815a760 con 0x7fa0580d04c0 2024-09-13T16:50:09.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa04f400700 1 -- 172.21.15.157:0/3690934524 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa0500046e0 con 0x7fa0580d04c0 2024-09-13T16:50:09.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa04f400700 1 -- 172.21.15.157:0/3690934524 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fa050022070 con 0x7fa0580d04c0 2024-09-13T16:50:09.606 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.600+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa058158610 con 0x7fa0580d04c0 2024-09-13T16:50:09.609 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.604+0000 7fa04f400700 1 -- 172.21.15.157:0/3690934524 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fa058158610 con 0x7fa0580d04c0 2024-09-13T16:50:09.621 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:09.616+0000 7f8ef30d62c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:50:09.659 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:09.653+0000 7fc3872eb2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:50:09.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.688+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "edeb379b-9a58-440e-911d-fd3119dd838d", "id": 7} v 0) v1 -- 0x7fa0580d08a0 con 0x7fa0580d04c0 2024-09-13T16:50:09.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa04f400700 1 -- 172.21.15.157:0/3690934524 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "edeb379b-9a58-440e-911d-fd3119dd838d", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa05002c060 con 0x7fa0580d04c0 2024-09-13T16:50:09.704 INFO:teuthology.orchestra.run.smithi157.stdout:7 2024-09-13T16:50:09.706 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0580d04c0 msgr2=0x7fa0580cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0580d04c0 0x7fa0580cbf70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b680 tx=0x7fa050004af0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 shutdown_connections 2024-09-13T16:50:09.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0580d04c0 0x7fa0580cbf70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580b46b0 0x7fa0581f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 --2- 172.21.15.157:0/3690934524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa05806c960 0x7fa05815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 >> 172.21.15.157:0/3690934524 conn(0x7fa0580b2910 msgr2=0x7fa05815c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:09.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 shutdown_connections 2024-09-13T16:50:09.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-13T16:50:09.700+0000 7fa05d200700 1 -- 172.21.15.157:0/3690934524 wait complete. 2024-09-13T16:50:09.725 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T16:50:09.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T16:50:09.731 INFO:tasks.ceph.osd.0:Started 2024-09-13T16:50:09.731 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T16:50:09.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T16:50:09.736 INFO:tasks.ceph.osd.1:Started 2024-09-13T16:50:09.736 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T16:50:09.737 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T16:50:09.740 INFO:tasks.ceph.osd.2:Started 2024-09-13T16:50:09.740 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T16:50:09.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T16:50:09.745 INFO:tasks.ceph.osd.3:Started 2024-09-13T16:50:09.745 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T16:50:09.745 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T16:50:09.776 INFO:tasks.ceph.osd.4:Started 2024-09-13T16:50:09.776 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T16:50:09.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T16:50:09.781 INFO:tasks.ceph.osd.5:Started 2024-09-13T16:50:09.781 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T16:50:09.781 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T16:50:09.786 INFO:tasks.ceph.osd.6:Started 2024-09-13T16:50:09.786 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T16:50:09.786 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T16:50:09.790 INFO:tasks.ceph.osd.7:Started 2024-09-13T16:50:09.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.965+0000 7f0fb8a00700 1 Processor -- start 2024-09-13T16:50:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb8a00700 1 -- start start 2024-09-13T16:50:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb8a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb406ff30 0x7f0fb40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb8a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb40cf070 con 0x7f0fb406fb50 2024-09-13T16:50:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb8a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb40b4be0 con 0x7f0fb40b4390 2024-09-13T16:50:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb8a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb415bc20 con 0x7f0fb406ff30 2024-09-13T16:50:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb3400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0fb406fb50 0x7f0fb40cfe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45814/0 (socket says 172.21.15.3:45814) 2024-09-13T16:50:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb3400700 1 -- 172.21.15.3:0/4170228479 learned_addr learned my addr 172.21.15.3:0/4170228479 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb2a00700 1 --2- 172.21.15.3:0/4170228479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb406ff30 0x7f0fb40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb2000700 1 -- 172.21.15.3:0/4170228479 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936405716 0 0) 0x7f0fb40cf070 con 0x7f0fb406fb50 2024-09-13T16:50:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb2000700 1 -- 172.21.15.3:0/4170228479 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f980035d0 con 0x7f0fb406fb50 2024-09-13T16:50:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb2a00700 1 -- 172.21.15.3:0/4170228479 >> v1:172.21.15.157:6789/0 conn(0x7f0fb40b4390 legacy=0x7f0fb415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb2a00700 1 -- 172.21.15.3:0/4170228479 >> v1:172.21.15.3:6789/0 conn(0x7f0fb406fb50 legacy=0x7f0fb40cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb2a00700 1 -- 172.21.15.3:0/4170228479 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb40cf330 con 0x7f0fb406ff30 2024-09-13T16:50:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.969+0000 7f0fb2a00700 1 --2- 172.21.15.3:0/4170228479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb406ff30 0x7f0fb40b3e50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009880 tx=0x7f0fa0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91611ed2b903023 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb2000700 1 -- 172.21.15.3:0/4170228479 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0011830 con 0x7f0fb406ff30 2024-09-13T16:50:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb2000700 1 -- 172.21.15.3:0/4170228479 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa0011990 con 0x7f0fb406ff30 2024-09-13T16:50:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb2000700 1 -- 172.21.15.3:0/4170228479 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0011d20 con 0x7f0fb406ff30 2024-09-13T16:50:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb8a00700 1 -- 172.21.15.3:0/4170228479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb406ff30 msgr2=0x7f0fb40b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb8a00700 1 --2- 172.21.15.3:0/4170228479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb406ff30 0x7f0fb40b3e50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009880 tx=0x7f0fa0009b90 comp rx=0 tx=0).stop 2024-09-13T16:50:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb8a00700 1 -- 172.21.15.3:0/4170228479 shutdown_connections 2024-09-13T16:50:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb8a00700 1 --2- 172.21.15.3:0/4170228479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb406ff30 0x7f0fb40b3e50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb8a00700 1 -- 172.21.15.3:0/4170228479 >> 172.21.15.3:0/4170228479 conn(0x7f0fb40b2910 msgr2=0x7f0fb40b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb8a00700 1 -- 172.21.15.3:0/4170228479 shutdown_connections 2024-09-13T16:50:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.973+0000 7f0fb8a00700 1 -- 172.21.15.3:0/4170228479 wait complete. 2024-09-13T16:50:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb8a00700 1 Processor -- start 2024-09-13T16:50:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb8a00700 1 -- start start 2024-09-13T16:50:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb8a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb406fb50 0x7f0fb415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb8a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb406ff30 0x7f0fb413c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb8a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb40b4390 0x7f0fb415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb8a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb415f5d0 con 0x7f0fb406ff30 2024-09-13T16:50:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb8a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb415dd10 con 0x7f0fb406fb50 2024-09-13T16:50:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb8a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb413cd80 con 0x7f0fb40b4390 2024-09-13T16:50:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb2a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb406ff30 0x7f0fb413c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb3e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb40b4390 0x7f0fb415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb3e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb40b4390 0x7f0fb415a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:39284/0 (socket says 172.21.15.3:39284) 2024-09-13T16:50:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb3e00700 1 -- 172.21.15.3:0/2507441687 learned_addr learned my addr 172.21.15.3:0/2507441687 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb3400700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb406fb50 0x7f0fb415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb2a00700 1 -- 172.21.15.3:0/2507441687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb40b4390 msgr2=0x7f0fb415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb2a00700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb40b4390 0x7f0fb415a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb2a00700 1 -- 172.21.15.3:0/2507441687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb406fb50 msgr2=0x7f0fb415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb2a00700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb406fb50 0x7f0fb415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb2a00700 1 -- 172.21.15.3:0/2507441687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa00094b0 con 0x7f0fb406ff30 2024-09-13T16:50:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.977+0000 7f0fb3400700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb406fb50 0x7f0fb415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fb2a00700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb406ff30 0x7f0fb413c840 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0015000 tx=0x7f0fa0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fab400700 1 -- 172.21.15.3:0/2507441687 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00036c0 con 0x7f0fb406ff30 2024-09-13T16:50:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fab400700 1 -- 172.21.15.3:0/2507441687 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa0003820 con 0x7f0fb406ff30 2024-09-13T16:50:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb415e530 con 0x7f0fb406ff30 2024-09-13T16:50:09.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fab400700 1 -- 172.21.15.3:0/2507441687 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0003b90 con 0x7f0fb406ff30 2024-09-13T16:50:09.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb415eaf0 con 0x7f0fb406ff30 2024-09-13T16:50:09.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fab400700 1 -- 172.21.15.3:0/2507441687 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0fa0025410 con 0x7f0fb406ff30 2024-09-13T16:50:09.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fab400700 1 -- 172.21.15.3:0/2507441687 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0fa001b400 con 0x7f0fb406ff30 2024-09-13T16:50:09.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.981+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb40b3e50 con 0x7f0fb406ff30 2024-09-13T16:50:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:09.985+0000 7f0fab400700 1 -- 172.21.15.3:0/2507441687 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f0fa002a020 con 0x7f0fb406ff30 2024-09-13T16:50:10.065 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:10.060+0000 7f8ef30d62c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:50:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.061+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fb41583b0 con 0x7f0fb406ff30 2024-09-13T16:50:10.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fab400700 1 -- 172.21.15.3:0/2507441687 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0fa00256c0 con 0x7f0fb406ff30 2024-09-13T16:50:10.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:10.069 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:09.694922+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb406ff30 msgr2=0x7f0fb413c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb406ff30 0x7f0fb413c840 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0015000 tx=0x7f0fa0004770 comp rx=0 tx=0).stop 2024-09-13T16:50:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 shutdown_connections 2024-09-13T16:50:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb40b4390 0x7f0fb415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb406ff30 0x7f0fb413c840 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 --2- 172.21.15.3:0/2507441687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb406fb50 0x7f0fb415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 >> 172.21.15.3:0/2507441687 conn(0x7f0fb40b2910 msgr2=0x7f0fb415b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 shutdown_connections 2024-09-13T16:50:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:10.065+0000 7f0fb8a00700 1 -- 172.21.15.3:0/2507441687 wait complete. 2024-09-13T16:50:10.092 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T16:50:10.092 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T16:50:10.146 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:10.141+0000 7fc3872eb2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:50:10.315 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:10.308+0000 7f8ef30d62c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:50:10.410 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:10.404+0000 7f8ef30d62c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:50:10.415 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-13T16:50:10.409+0000 7f8dce24d880 -1 Falling back to public interface 2024-09-13T16:50:10.420 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:10.417+0000 7fc3872eb2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:50:10.426 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-13T16:50:10.421+0000 7f71fb9b6880 -1 Falling back to public interface 2024-09-13T16:50:10.429 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-13T16:50:10.425+0000 7f6eaf53c880 -1 Falling back to public interface 2024-09-13T16:50:10.438 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-09-13T16:50:10.432+0000 7f9d604fd880 -1 Falling back to public interface 2024-09-13T16:50:10.446 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-13T16:50:10.441+0000 7f7284616880 -1 Falling back to public interface 2024-09-13T16:50:10.458 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-09-13T16:50:10.452+0000 7f6b4ed7e880 -1 Falling back to public interface 2024-09-13T16:50:10.462 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-09-13T16:50:10.456+0000 7f2190005880 -1 Falling back to public interface 2024-09-13T16:50:10.477 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-09-13T16:50:10.472+0000 7f086c637880 -1 Falling back to public interface 2024-09-13T16:50:10.530 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:10.525+0000 7fc3872eb2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:50:10.622 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:10.616+0000 7f8ef30d62c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:50:10.760 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:10.753+0000 7fc3872eb2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:50:10.898 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:11.049 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff780e00700 1 Processor -- start 2024-09-13T16:50:11.049 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff780e00700 1 -- start start 2024-09-13T16:50:11.051 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff780e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff77c0d1980 0x7ff77c0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:11.051 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff780e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c0cd590 con 0x7ff77c0cec80 2024-09-13T16:50:11.051 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff780e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c0cd870 con 0x7ff77c1600e0 2024-09-13T16:50:11.051 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff780e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c0d2330 con 0x7ff77c0d1980 2024-09-13T16:50:11.051 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff77be00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff77c0cec80 0x7ff77c0cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45824/0 (socket says 172.21.15.3:45824) 2024-09-13T16:50:11.051 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff77be00700 1 -- 172.21.15.3:0/236993965 learned_addr learned my addr 172.21.15.3:0/236993965 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:11.052 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.045+0000 7ff77aa00700 1 --2- 172.21.15.3:0/236993965 >> v2:172.21.15.3:3300/0 conn(0x7ff77c0d1980 0x7ff77c0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:11.052 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff77aa00700 1 -- 172.21.15.3:0/236993965 >> v1:172.21.15.157:6789/0 conn(0x7ff77c1600e0 legacy=0x7ff77c0b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:11.052 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff77aa00700 1 -- 172.21.15.3:0/236993965 >> v1:172.21.15.3:6789/0 conn(0x7ff77c0cec80 legacy=0x7ff77c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:11.052 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff77aa00700 1 -- 172.21.15.3:0/236993965 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0cdb50 con 0x7ff77c0d1980 2024-09-13T16:50:11.052 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff77a000700 1 -- 172.21.15.3:0/236993965 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835163757 0 0) 0x7ff77c0cd590 con 0x7ff77c0cec80 2024-09-13T16:50:11.053 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff77aa00700 1 --2- 172.21.15.3:0/236993965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c0d1980 0x7ff77c0d1df0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff768009850 tx=0x7ff768009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377467b018b5c50e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:11.053 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff77a000700 1 -- 172.21.15.3:0/236993965 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768011830 con 0x7ff77c0d1980 2024-09-13T16:50:11.053 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff77a000700 1 -- 172.21.15.3:0/236993965 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff768011990 con 0x7ff77c0d1980 2024-09-13T16:50:11.053 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff77a000700 1 -- 172.21.15.3:0/236993965 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768011d20 con 0x7ff77c0d1980 2024-09-13T16:50:11.054 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff780e00700 1 -- 172.21.15.3:0/236993965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c0d1980 msgr2=0x7ff77c0d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:11.054 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff780e00700 1 --2- 172.21.15.3:0/236993965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c0d1980 0x7ff77c0d1df0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff768009850 tx=0x7ff768009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:11.054 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff780e00700 1 -- 172.21.15.3:0/236993965 shutdown_connections 2024-09-13T16:50:11.055 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff780e00700 1 --2- 172.21.15.3:0/236993965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c0d1980 0x7ff77c0d1df0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:11.055 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff780e00700 1 -- 172.21.15.3:0/236993965 >> 172.21.15.3:0/236993965 conn(0x7ff77c0cf820 msgr2=0x7ff77c0cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:11.055 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff780e00700 1 -- 172.21.15.3:0/236993965 shutdown_connections 2024-09-13T16:50:11.055 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.049+0000 7ff780e00700 1 -- 172.21.15.3:0/236993965 wait complete. 2024-09-13T16:50:11.057 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff780e00700 1 Processor -- start 2024-09-13T16:50:11.057 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff780e00700 1 -- start start 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff780e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0cec80 0x7ff77c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff780e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c1600e0 0x7ff77c0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff780e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c15c050 0x7ff77c1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff780e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15a100 con 0x7ff77c15c050 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff780e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15a3e0 con 0x7ff77c0cec80 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff780e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15a6c0 con 0x7ff77c1600e0 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0cec80 0x7ff77c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0cec80 0x7ff77c159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:55872/0 (socket says 172.21.15.3:55872) 2024-09-13T16:50:11.059 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c15c050 0x7ff77c1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:11.060 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 -- 172.21.15.3:0/3181831021 learned_addr learned my addr 172.21.15.3:0/3181831021 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:11.060 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77aa00700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c1600e0 0x7ff77c0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:11.061 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 -- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c1600e0 msgr2=0x7ff77c0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:11.061 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c1600e0 0x7ff77c0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:11.061 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 -- 172.21.15.3:0/3181831021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c15c050 msgr2=0x7ff77c1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:11.061 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c15c050 0x7ff77c1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:11.061 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 -- 172.21.15.3:0/3181831021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680094b0 con 0x7ff77c0cec80 2024-09-13T16:50:11.061 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77aa00700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c1600e0 0x7ff77c0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:11.061 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.053+0000 7ff77b400700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0cec80 0x7ff77c159b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00c500 tx=0x7ff76c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:11.062 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.057+0000 7ff773400700 1 -- 172.21.15.3:0/3181831021 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0052d0 con 0x7ff77c0cec80 2024-09-13T16:50:11.062 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.057+0000 7ff773400700 1 -- 172.21.15.3:0/3181831021 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c005430 con 0x7ff77c0cec80 2024-09-13T16:50:11.062 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.057+0000 7ff773400700 1 -- 172.21.15.3:0/3181831021 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0055a0 con 0x7ff77c0cec80 2024-09-13T16:50:11.062 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.057+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c15a9a0 con 0x7ff77c0cec80 2024-09-13T16:50:11.062 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.057+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c15dda0 con 0x7ff77c0cec80 2024-09-13T16:50:11.062 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.057+0000 7ff773400700 1 -- 172.21.15.3:0/3181831021 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff76c003be0 con 0x7ff77c0cec80 2024-09-13T16:50:11.062 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.057+0000 7ff773400700 1 -- 172.21.15.3:0/3181831021 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff76c011e40 con 0x7ff77c0cec80 2024-09-13T16:50:11.062 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.057+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c15aec0 con 0x7ff77c0cec80 2024-09-13T16:50:11.066 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.061+0000 7ff773400700 1 -- 172.21.15.3:0/3181831021 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff77c15aec0 con 0x7ff77c0cec80 2024-09-13T16:50:11.138 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.133+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff77c0d1df0 con 0x7ff77c0cec80 2024-09-13T16:50:11.140 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.133+0000 7ff773400700 1 -- 172.21.15.3:0/3181831021 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff76c017070 con 0x7ff77c0cec80 2024-09-13T16:50:11.140 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-13T16:50:11.140 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:09.694922+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:11.141 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0cec80 msgr2=0x7ff77c159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:11.141 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0cec80 0x7ff77c159b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00c500 tx=0x7ff76c00c810 comp rx=0 tx=0).stop 2024-09-13T16:50:11.141 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 shutdown_connections 2024-09-13T16:50:11.141 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77c15c050 0x7ff77c1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:11.141 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c1600e0 0x7ff77c0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:11.142 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 --2- 172.21.15.3:0/3181831021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0cec80 0x7ff77c159b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:11.142 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 >> 172.21.15.3:0/3181831021 conn(0x7ff77c0cf820 msgr2=0x7ff77c0b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:11.142 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 shutdown_connections 2024-09-13T16:50:11.142 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:11.137+0000 7ff780e00700 1 -- 172.21.15.3:0/3181831021 wait complete. 2024-09-13T16:50:11.158 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T16:50:11.332 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:11.328+0000 7f8ef30d62c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:50:11.425 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:11.420+0000 7f8ef30d62c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:50:11.452 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:11.445+0000 7fc3872eb2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:50:11.552 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:11.545+0000 7fc3872eb2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:50:11.635 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:11.628+0000 7f8ef30d62c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:50:11.729 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:11.724+0000 7f8ef30d62c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:50:11.734 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:11.729+0000 7fc3872eb2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:50:11.829 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:11.824+0000 7f8ef30d62c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:50:11.830 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:11.825+0000 7fc3872eb2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:50:11.937 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:11.933+0000 7fc3872eb2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:50:12.225 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:12.220+0000 7f8ef30d62c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:50:12.322 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:12.316+0000 7f8ef30d62c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:50:12.335 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:12.329+0000 7fc3872eb2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:50:12.443 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:12.437+0000 7fc3872eb2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:50:13.161 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:13.156+0000 7f8ef30d62c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:50:13.333 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-13T16:50:13.329+0000 7fc3872eb2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:50:14.199 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:14.192+0000 7f8ed9800700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T16:50:14.200 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-13T16:50:14.196+0000 7f8ed9800700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T16:50:14.438 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-13T16:50:14.433+0000 7f71fb9b6880 -1 osd.2 0 log_to_monitors true 2024-09-13T16:50:14.463 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-09-13T16:50:14.456+0000 7f9d604fd880 -1 osd.5 0 log_to_monitors true 2024-09-13T16:50:14.467 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-13T16:50:14.461+0000 7f7284616880 -1 osd.0 0 log_to_monitors true 2024-09-13T16:50:14.475 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-13T16:50:14.469+0000 7f8dce24d880 -1 osd.1 0 log_to_monitors true 2024-09-13T16:50:14.521 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-09-13T16:50:14.516+0000 7f6b4ed7e880 -1 osd.4 0 log_to_monitors true 2024-09-13T16:50:14.525 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-09-13T16:50:14.520+0000 7f2190005880 -1 osd.7 0 log_to_monitors true 2024-09-13T16:50:14.530 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-09-13T16:50:14.524+0000 7f086c637880 -1 osd.6 0 log_to_monitors true 2024-09-13T16:50:14.533 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-13T16:50:14.529+0000 7f6eaf53c880 -1 osd.3 0 log_to_monitors true 2024-09-13T16:50:17.964 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:17.969 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-09-13T16:50:17.964+0000 7f218ae00700 -1 osd.7 0 waiting for initial osdmap 2024-09-13T16:50:17.986 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-09-13T16:50:17.980+0000 7f0867600700 -1 osd.6 0 waiting for initial osdmap 2024-09-13T16:50:17.988 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-09-13T16:50:17.984+0000 7f2180a00700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:50:17.991 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-09-13T16:50:17.988+0000 7f085d200700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:50:17.993 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-09-13T16:50:17.988+0000 7f9d5b400700 -1 osd.5 0 waiting for initial osdmap 2024-09-13T16:50:17.997 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-09-13T16:50:17.992+0000 7f9d51000700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:50:18.031 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-09-13T16:50:18.024+0000 7f6b49c00700 -1 osd.4 0 waiting for initial osdmap 2024-09-13T16:50:18.039 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-09-13T16:50:18.032+0000 7f6b3f800700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:50:18.156 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.149+0000 7f7f07200700 1 Processor -- start 2024-09-13T16:50:18.156 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.149+0000 7f7f07200700 1 -- start start 2024-09-13T16:50:18.157 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f07200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f08144570 0x7f7f080b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:18.157 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f07200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f080d1220 0x7f7f0815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:18.159 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f06800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f080d1220 0x7f7f0815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:18.159 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f07200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f080b5ce0 con 0x7f7f080cffb0 2024-09-13T16:50:18.159 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f07200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080b0a60 con 0x7f7f080d1220 2024-09-13T16:50:18.159 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f07200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080b0d40 con 0x7f7f08144570 2024-09-13T16:50:18.159 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f05400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7f080cffb0 0x7f7f080d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:49310/0 (socket says 172.21.15.3:49310) 2024-09-13T16:50:18.159 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f05400700 1 -- 172.21.15.3:0/647241075 learned_addr learned my addr 172.21.15.3:0/647241075 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:18.159 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.153+0000 7f7f05e00700 1 --2- 172.21.15.3:0/647241075 >> v2:172.21.15.157:3300/0 conn(0x7f7f08144570 0x7f7f080b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:18.160 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f05e00700 1 -- 172.21.15.3:0/647241075 >> v2:172.21.15.3:3300/0 conn(0x7f7f080d1220 msgr2=0x7f7f0815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:18.160 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f05e00700 1 --2- 172.21.15.3:0/647241075 >> v2:172.21.15.3:3300/0 conn(0x7f7f080d1220 0x7f7f0815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.160 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f05e00700 1 -- 172.21.15.3:0/647241075 >> v1:172.21.15.157:6789/0 conn(0x7f7f080cffb0 legacy=0x7f7f080d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:18.160 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f05e00700 1 -- 172.21.15.3:0/647241075 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080b1020 con 0x7f7f08144570 2024-09-13T16:50:18.160 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f06800700 1 --2- 172.21.15.3:0/647241075 >> v2:172.21.15.3:3300/0 conn(0x7f7f080d1220 0x7f7f0815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:18.161 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f05e00700 1 --2- 172.21.15.3:0/647241075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08144570 0x7f7f080b5370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7f0806ae30 tx=0x7f7ef4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57387a831d075ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:18.162 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f04a00700 1 -- 172.21.15.3:0/647241075 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4004230 con 0x7f7f08144570 2024-09-13T16:50:18.162 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f04a00700 1 -- 172.21.15.3:0/647241075 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4004390 con 0x7f7f08144570 2024-09-13T16:50:18.162 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f04a00700 1 -- 172.21.15.3:0/647241075 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40036c0 con 0x7f7f08144570 2024-09-13T16:50:18.163 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f07200700 1 -- 172.21.15.3:0/647241075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08144570 msgr2=0x7f7f080b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:18.163 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.157+0000 7f7f07200700 1 --2- 172.21.15.3:0/647241075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08144570 0x7f7f080b5370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7f0806ae30 tx=0x7f7ef4009b30 comp rx=0 tx=0).stop 2024-09-13T16:50:18.164 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.161+0000 7f7f07200700 1 -- 172.21.15.3:0/647241075 shutdown_connections 2024-09-13T16:50:18.164 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.161+0000 7f7f07200700 1 --2- 172.21.15.3:0/647241075 >> v2:172.21.15.3:3300/0 conn(0x7f7f080d1220 0x7f7f0815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.164 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.161+0000 7f7f07200700 1 --2- 172.21.15.3:0/647241075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f08144570 0x7f7f080b5370 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.164 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.161+0000 7f7f07200700 1 -- 172.21.15.3:0/647241075 >> 172.21.15.3:0/647241075 conn(0x7f7f080721d0 msgr2=0x7f7f0806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:18.165 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.161+0000 7f7f07200700 1 -- 172.21.15.3:0/647241075 shutdown_connections 2024-09-13T16:50:18.165 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.161+0000 7f7f07200700 1 -- 172.21.15.3:0/647241075 wait complete. 2024-09-13T16:50:18.167 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.161+0000 7f7f07200700 1 Processor -- start 2024-09-13T16:50:18.167 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.161+0000 7f7f07200700 1 -- start start 2024-09-13T16:50:18.169 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f07200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f080cffb0 0x7f7f081ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:18.170 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f07200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f080d1220 0x7f7f081ec030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:18.170 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f07200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f08144570 0x7f7f081ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:18.171 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f07200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0813dd30 con 0x7f7f080cffb0 2024-09-13T16:50:18.171 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f07200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0813b350 con 0x7f7f080d1220 2024-09-13T16:50:18.171 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f07200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0813d0b0 con 0x7f7f08144570 2024-09-13T16:50:18.171 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f05e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f080cffb0 0x7f7f081ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:18.171 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f06800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f08144570 0x7f7f081ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:18.171 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f06800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f08144570 0x7f7f081ffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:44090/0 (socket says 172.21.15.3:44090) 2024-09-13T16:50:18.171 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f06800700 1 -- 172.21.15.3:0/1866444432 learned_addr learned my addr 172.21.15.3:0/1866444432 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:18.171 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.165+0000 7f7f05400700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f080d1220 0x7f7f081ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:18.172 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7f05400700 1 -- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f08144570 msgr2=0x7f7f081ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:18.172 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7f05400700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f08144570 0x7f7f081ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.172 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7f05400700 1 -- 172.21.15.3:0/1866444432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f080cffb0 msgr2=0x7f7f081ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:18.172 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7f05400700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f080cffb0 0x7f7f081ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.172 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7f05400700 1 -- 172.21.15.3:0/1866444432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f00009a70 con 0x7f7f080d1220 2024-09-13T16:50:18.172 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7f05400700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f080d1220 0x7f7f081ec030 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800c7b0 tx=0x7f7ef800cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:18.173 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7efe000700 1 -- 172.21.15.3:0/1866444432 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8004ca0 con 0x7f7f080d1220 2024-09-13T16:50:18.173 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7efe000700 1 -- 172.21.15.3:0/1866444432 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef8004e00 con 0x7f7f080d1220 2024-09-13T16:50:18.175 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7efe000700 1 -- 172.21.15.3:0/1866444432 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8007310 con 0x7f7f080d1220 2024-09-13T16:50:18.175 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef40094b0 con 0x7f7f080d1220 2024-09-13T16:50:18.175 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.169+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0813ad00 con 0x7f7f080d1220 2024-09-13T16:50:18.177 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.173+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0813ab60 con 0x7f7f080d1220 2024-09-13T16:50:18.180 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.173+0000 7f7efe000700 1 -- 172.21.15.3:0/1866444432 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f7ef8007470 con 0x7f7f080d1220 2024-09-13T16:50:18.180 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.177+0000 7f7efe000700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7ee80666f0 0x7f7ee8068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:18.181 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.177+0000 7f7efe000700 1 -- 172.21.15.3:0/1866444432 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f7ef8082f80 con 0x7f7f080d1220 2024-09-13T16:50:18.181 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.177+0000 7f7f05e00700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7ee80666f0 0x7f7ee8068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:18.183 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.177+0000 7f7f05e00700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7ee80666f0 0x7f7ee8068bb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7f0806ae30 tx=0x7f7ef4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:18.198 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.193+0000 7f7efe000700 1 -- 172.21.15.3:0/1866444432 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7f0813ab60 con 0x7f7f080d1220 2024-09-13T16:50:18.542 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.537+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f0813ab60 con 0x7f7f080d1220 2024-09-13T16:50:18.543 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.537+0000 7f7efe000700 1 -- 172.21.15.3:0/1866444432 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f7f0813ab60 con 0x7f7f080d1220 2024-09-13T16:50:18.543 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-13T16:50:18.543 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":12,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:16.235505+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:18.547 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7ee80666f0 msgr2=0x7f7ee8068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:18.547 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7ee80666f0 0x7f7ee8068bb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7f0806ae30 tx=0x7f7ef4008030 comp rx=0 tx=0).stop 2024-09-13T16:50:18.547 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f080d1220 msgr2=0x7f7f081ec030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:18.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f080d1220 0x7f7f081ec030 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800c7b0 tx=0x7f7ef800cac0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 shutdown_connections 2024-09-13T16:50:18.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7ee80666f0 0x7f7ee8068bb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f08144570 0x7f7f081ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f080d1220 0x7f7f081ec030 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 --2- 172.21.15.3:0/1866444432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f080cffb0 0x7f7f081ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:18.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.541+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 >> 172.21.15.3:0/1866444432 conn(0x7f7f080721d0 msgr2=0x7f7f080b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:18.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.545+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 shutdown_connections 2024-09-13T16:50:18.549 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:18.545+0000 7f7f07200700 1 -- 172.21.15.3:0/1866444432 wait complete. 2024-09-13T16:50:18.576 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T16:50:18.713 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-13T16:50:18.709+0000 7f71f6800700 -1 osd.2 0 waiting for initial osdmap 2024-09-13T16:50:18.718 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-13T16:50:18.713+0000 7f727f400700 -1 osd.0 0 waiting for initial osdmap 2024-09-13T16:50:18.725 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-13T16:50:18.721+0000 7f6eaa400700 -1 osd.3 0 waiting for initial osdmap 2024-09-13T16:50:18.726 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-13T16:50:18.721+0000 7f71ec400700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:50:18.726 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-13T16:50:18.721+0000 7f7275000700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:50:18.730 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-13T16:50:18.725+0000 7f6ea0000700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:50:18.748 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-13T16:50:18.741+0000 7f8dc9200700 -1 osd.1 0 waiting for initial osdmap 2024-09-13T16:50:18.757 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-13T16:50:18.753+0000 7f8dbee00700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:50:25.384 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:25.548 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.545+0000 7f16c2400700 1 Processor -- start 2024-09-13T16:50:25.549 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.545+0000 7f16c2400700 1 -- start start 2024-09-13T16:50:25.552 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.545+0000 7f16c2400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16bc1438b0 0x7f16bc0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:25.553 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16c2400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:25.553 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16c2400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16bc0cda10 con 0x7f16bc06b010 2024-09-13T16:50:25.553 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16c2400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16bc0cd730 con 0x7f16bc1438b0 2024-09-13T16:50:25.553 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16c2400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc06a530 con 0x7f16bc0b2200 2024-09-13T16:50:25.553 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16baa00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:25.553 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16baa00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc0b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:54634/0 (socket says 172.21.15.3:54634) 2024-09-13T16:50:25.553 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16baa00700 1 -- 172.21.15.3:0/652713552 learned_addr learned my addr 172.21.15.3:0/652713552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:25.554 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16bb400700 1 --2- 172.21.15.3:0/652713552 >> v2:172.21.15.157:3300/0 conn(0x7f16bc1438b0 0x7f16bc0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:25.554 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16baa00700 1 -- 172.21.15.3:0/652713552 >> v2:172.21.15.157:3300/0 conn(0x7f16bc1438b0 msgr2=0x7f16bc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:25.554 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16baa00700 1 --2- 172.21.15.3:0/652713552 >> v2:172.21.15.157:3300/0 conn(0x7f16bc1438b0 0x7f16bc0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.554 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16baa00700 1 -- 172.21.15.3:0/652713552 >> v1:172.21.15.157:6789/0 conn(0x7f16bc06b010 legacy=0x7f16bc06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:25.554 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16baa00700 1 -- 172.21.15.3:0/652713552 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc06a810 con 0x7f16bc0b2200 2024-09-13T16:50:25.554 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16ba000700 1 -- 172.21.15.3:0/652713552 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083455707 0 0) 0x7f16bc0cda10 con 0x7f16bc06b010 2024-09-13T16:50:25.555 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16baa00700 1 --2- 172.21.15.3:0/652713552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc0b4b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009910 tx=0x7f16a8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b6ae34a9f53f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:25.555 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16ba000700 1 -- 172.21.15.3:0/652713552 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8011830 con 0x7f16bc0b2200 2024-09-13T16:50:25.556 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16ba000700 1 -- 172.21.15.3:0/652713552 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8011990 con 0x7f16bc0b2200 2024-09-13T16:50:25.556 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.549+0000 7f16ba000700 1 -- 172.21.15.3:0/652713552 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8011d40 con 0x7f16bc0b2200 2024-09-13T16:50:25.557 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.553+0000 7f16c2400700 1 -- 172.21.15.3:0/652713552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 msgr2=0x7f16bc0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:25.557 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.553+0000 7f16c2400700 1 --2- 172.21.15.3:0/652713552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc0b4b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009910 tx=0x7f16a8009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:25.557 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.553+0000 7f16c2400700 1 -- 172.21.15.3:0/652713552 shutdown_connections 2024-09-13T16:50:25.557 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.553+0000 7f16c2400700 1 --2- 172.21.15.3:0/652713552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc0b4b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.557 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.553+0000 7f16c2400700 1 --2- 172.21.15.3:0/652713552 >> v2:172.21.15.157:3300/0 conn(0x7f16bc1438b0 0x7f16bc0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.557 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.553+0000 7f16c2400700 1 -- 172.21.15.3:0/652713552 >> 172.21.15.3:0/652713552 conn(0x7f16bc0708b0 msgr2=0x7f16bc0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:25.558 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.553+0000 7f16c2400700 1 -- 172.21.15.3:0/652713552 shutdown_connections 2024-09-13T16:50:25.558 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.553+0000 7f16c2400700 1 -- 172.21.15.3:0/652713552 wait complete. 2024-09-13T16:50:25.560 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16c2400700 1 Processor -- start 2024-09-13T16:50:25.561 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16c2400700 1 -- start start 2024-09-13T16:50:25.562 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16c2400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16bc06b010 0x7f16bc15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:25.562 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16c2400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc137330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:25.562 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16c2400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16bc1438b0 0x7f16bc13f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:25.562 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16c2400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc137d40 con 0x7f16bc06b010 2024-09-13T16:50:25.563 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16c2400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc138020 con 0x7f16bc1438b0 2024-09-13T16:50:25.563 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16c2400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc138300 con 0x7f16bc0b2200 2024-09-13T16:50:25.563 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bb400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16bc06b010 0x7f16bc15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:25.563 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bb400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16bc06b010 0x7f16bc15a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41108/0 (socket says 172.21.15.3:41108) 2024-09-13T16:50:25.563 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bb400700 1 -- 172.21.15.3:0/2060675232 learned_addr learned my addr 172.21.15.3:0/2060675232 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bbe00700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16bc1438b0 0x7f16bc13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16baa00700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc137330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bb400700 1 -- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 msgr2=0x7f16bc137330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bb400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc137330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bb400700 1 -- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16bc1438b0 msgr2=0x7f16bc13f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bb400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16bc1438b0 0x7f16bc13f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16bb400700 1 -- 172.21.15.3:0/2060675232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8009570 con 0x7f16bc06b010 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.557+0000 7f16baa00700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc137330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:25.564 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.561+0000 7f16bb400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16bc06b010 0x7f16bc15a6b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00e840 tx=0x7f16ac00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:25.565 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.561+0000 7f16b3400700 1 -- 172.21.15.3:0/2060675232 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0095a0 con 0x7f16bc06b010 2024-09-13T16:50:25.565 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.561+0000 7f16b3400700 1 -- 172.21.15.3:0/2060675232 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac013680 con 0x7f16bc06b010 2024-09-13T16:50:25.565 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.561+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc1385e0 con 0x7f16bc06b010 2024-09-13T16:50:25.565 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.561+0000 7f16b3400700 1 -- 172.21.15.3:0/2060675232 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0139f0 con 0x7f16bc06b010 2024-09-13T16:50:25.565 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.561+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc136b60 con 0x7f16bc06b010 2024-09-13T16:50:25.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.561+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc0b5390 con 0x7f16bc06b010 2024-09-13T16:50:25.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.565+0000 7f16b3400700 1 -- 172.21.15.3:0/2060675232 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f16ac013b50 con 0x7f16bc06b010 2024-09-13T16:50:25.571 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.565+0000 7f16b3400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f169c066730 0x7f169c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:25.571 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.565+0000 7f16b3400700 1 -- 172.21.15.3:0/2060675232 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f16ac083d70 con 0x7f16bc06b010 2024-09-13T16:50:25.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.569+0000 7f16baa00700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f169c066730 0x7f169c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:25.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.569+0000 7f16baa00700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f169c066730 0x7f169c068bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0b2600 tx=0x7f16a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:25.579 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.573+0000 7f16b3400700 1 -- 172.21.15.3:0/2060675232 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16ac051420 con 0x7f16bc06b010 2024-09-13T16:50:25.769 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.765+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16bc0c9d90 con 0x7f16bc06b010 2024-09-13T16:50:25.770 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.765+0000 7f16b3400700 1 -- 172.21.15.3:0/2060675232 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7f16ac054a40 con 0x7f16bc06b010 2024-09-13T16:50:25.770 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-13T16:50:25.771 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":16,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:21.998415+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f169c066730 msgr2=0x7f169c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f169c066730 0x7f169c068bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0b2600 tx=0x7f16a8008030 comp rx=0 tx=0).stop 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16bc06b010 msgr2=0x7f16bc15a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16bc06b010 0x7f16bc15a6b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00e840 tx=0x7f16ac00eb50 comp rx=0 tx=0).stop 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 shutdown_connections 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f169c066730 0x7f169c068bf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16bc1438b0 0x7f16bc13f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16bc0b2200 0x7f16bc137330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 --2- 172.21.15.3:0/2060675232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16bc06b010 0x7f16bc15a6b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:25.774 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 >> 172.21.15.3:0/2060675232 conn(0x7f16bc0708b0 msgr2=0x7f16bc06b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:25.775 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 shutdown_connections 2024-09-13T16:50:25.775 INFO:teuthology.misc.health.smithi003.stderr:2024-09-13T16:50:25.769+0000 7f16c2400700 1 -- 172.21.15.3:0/2060675232 wait complete. 2024-09-13T16:50:25.792 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T16:50:25.792 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T16:50:25.792 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:50:25.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:50:25.798 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:50:25.798 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T16:50:25.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T16:50:25.803 INFO:tasks.ceph.mds.c:Started 2024-09-13T16:50:25.803 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:50:25.803 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:50:25.809 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:50:25.809 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:50:25.809 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:50:25.814 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:50:25.814 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T16:50:25.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:25.913 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-13T16:50:25.918 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-13T16:50:25.928 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-13T16:50:25.930 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-13T16:50:25.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe879000700 1 Processor -- start 2024-09-13T16:50:25.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe879000700 1 -- start start 2024-09-13T16:50:25.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe879000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8741443b0 0x7fe8740b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe879000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740ce8e0 con 0x7fe87406ccf0 2024-09-13T16:50:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe879000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740ccd30 con 0x7fe874144790 2024-09-13T16:50:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe879000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740cd010 con 0x7fe8741443b0 2024-09-13T16:50:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe873e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe87406ccf0 0x7fe87415b220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42116/0 (socket says 172.21.15.3:42116) 2024-09-13T16:50:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe873e00700 1 -- 172.21.15.3:0/3501452210 learned_addr learned my addr 172.21.15.3:0/3501452210 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.977+0000 7fe873400700 1 --2- 172.21.15.3:0/3501452210 >> v2:172.21.15.3:3300/0 conn(0x7fe8741443b0 0x7fe8740b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe872000700 1 -- 172.21.15.3:0/3501452210 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248662793 0 0) 0x7fe8740ce8e0 con 0x7fe87406ccf0 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe872000700 1 -- 172.21.15.3:0/3501452210 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8580035d0 con 0x7fe87406ccf0 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe872000700 1 -- 172.21.15.3:0/3501452210 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360326616 0 0) 0x7fe8740ccd30 con 0x7fe874144790 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe872000700 1 -- 172.21.15.3:0/3501452210 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8740ce8e0 con 0x7fe874144790 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe873400700 1 -- 172.21.15.3:0/3501452210 >> v1:172.21.15.157:6789/0 conn(0x7fe874144790 legacy=0x7fe87406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe873400700 1 -- 172.21.15.3:0/3501452210 >> v1:172.21.15.3:6789/0 conn(0x7fe87406ccf0 legacy=0x7fe87415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe873400700 1 -- 172.21.15.3:0/3501452210 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740cd2f0 con 0x7fe8741443b0 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe873400700 1 --2- 172.21.15.3:0/3501452210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8741443b0 0x7fe8740b39f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe864009850 tx=0x7fe864009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875ba3a0053a82a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe872000700 1 -- 172.21.15.3:0/3501452210 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864011830 con 0x7fe8741443b0 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe872000700 1 -- 172.21.15.3:0/3501452210 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864011990 con 0x7fe8741443b0 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe872000700 1 -- 172.21.15.3:0/3501452210 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864011d40 con 0x7fe8741443b0 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe879000700 1 -- 172.21.15.3:0/3501452210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8741443b0 msgr2=0x7fe8740b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe879000700 1 --2- 172.21.15.3:0/3501452210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8741443b0 0x7fe8740b39f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe864009850 tx=0x7fe864009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe879000700 1 -- 172.21.15.3:0/3501452210 shutdown_connections 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe879000700 1 --2- 172.21.15.3:0/3501452210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8741443b0 0x7fe8740b39f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe879000700 1 -- 172.21.15.3:0/3501452210 >> 172.21.15.3:0/3501452210 conn(0x7fe8740b0cf0 msgr2=0x7fe8740b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe879000700 1 -- 172.21.15.3:0/3501452210 shutdown_connections 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.981+0000 7fe879000700 1 -- 172.21.15.3:0/3501452210 wait complete. 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.985+0000 7fe879000700 1 Processor -- start 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.985+0000 7fe879000700 1 -- start start 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.985+0000 7fe879000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe87406ccf0 0x7fe87413f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.985+0000 7fe879000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8741443b0 0x7fe874132220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.985+0000 7fe879000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe87413e8f0 0x7fe87413d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.985+0000 7fe879000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87413fca0 con 0x7fe87413e8f0 2024-09-13T16:50:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.985+0000 7fe879000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe874132760 con 0x7fe87406ccf0 2024-09-13T16:50:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.985+0000 7fe879000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87413e460 con 0x7fe8741443b0 2024-09-13T16:50:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe87413e8f0 0x7fe87413d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe87406ccf0 0x7fe87413f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe87413e8f0 0x7fe87413d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41144/0 (socket says 172.21.15.3:41144) 2024-09-13T16:50:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 -- 172.21.15.3:0/2143884240 learned_addr learned my addr 172.21.15.3:0/2143884240 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe872a00700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8741443b0 0x7fe874132220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 -- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8741443b0 msgr2=0x7fe874132220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8741443b0 0x7fe874132220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 -- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe87406ccf0 msgr2=0x7fe87413f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe87406ccf0 0x7fe87413f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 -- 172.21.15.3:0/2143884240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8640094b0 con 0x7fe87413e8f0 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe873e00700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe87413e8f0 0x7fe87413d020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00cad0 tx=0x7fe86c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe86b400700 1 -- 172.21.15.3:0/2143884240 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c007480 con 0x7fe87413e8f0 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe86b400700 1 -- 172.21.15.3:0/2143884240 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c007600 con 0x7fe87413e8f0 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe86b400700 1 -- 172.21.15.3:0/2143884240 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0079b0 con 0x7fe87413e8f0 2024-09-13T16:50:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87413d850 con 0x7fe87413e8f0 2024-09-13T16:50:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87413de10 con 0x7fe87413e8f0 2024-09-13T16:50:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.989+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87413e5f0 con 0x7fe87413e8f0 2024-09-13T16:50:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.993+0000 7fe86b400700 1 -- 172.21.15.3:0/2143884240 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe86c01b3b0 con 0x7fe87413e8f0 2024-09-13T16:50:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.993+0000 7fe86b400700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe854066730 0x7fe854068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.993+0000 7fe86b400700 1 -- 172.21.15.3:0/2143884240 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe86c083750 con 0x7fe87413e8f0 2024-09-13T16:50:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.997+0000 7fe873400700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe854066730 0x7fe854068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:25.997+0000 7fe873400700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe854066730 0x7fe854068bf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe8640090b0 tx=0x7fe864008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.001+0000 7fe86b400700 1 -- 172.21.15.3:0/2143884240 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe86c050d70 con 0x7fe87413e8f0 2024-09-13T16:50:26.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.189+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8740b39f0 con 0x7fe87413e8f0 2024-09-13T16:50:26.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.193+0000 7fe86b400700 1 -- 172.21.15.3:0/2143884240 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7fe86c017090 con 0x7fe87413e8f0 2024-09-13T16:50:26.197 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:26.197 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:21.998415+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe854066730 msgr2=0x7fe854068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe854066730 0x7fe854068bf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe8640090b0 tx=0x7fe864008030 comp rx=0 tx=0).stop 2024-09-13T16:50:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe87413e8f0 msgr2=0x7fe87413d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe87413e8f0 0x7fe87413d020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00cad0 tx=0x7fe86c00cb00 comp rx=0 tx=0).stop 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 shutdown_connections 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe854066730 0x7fe854068bf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe87413e8f0 0x7fe87413d020 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8741443b0 0x7fe874132220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 --2- 172.21.15.3:0/2143884240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe87406ccf0 0x7fe87413f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 >> 172.21.15.3:0/2143884240 conn(0x7fe8740b0cf0 msgr2=0x7fe874140880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 shutdown_connections 2024-09-13T16:50:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.197+0000 7fe879000700 1 -- 172.21.15.3:0/2143884240 wait complete. 2024-09-13T16:50:26.223 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:50:26.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:50:26.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.401+0000 7f2030a00700 1 Processor -- start 2024-09-13T16:50:26.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.401+0000 7f2030a00700 1 -- start start 2024-09-13T16:50:26.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2030a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f202c0cde70 0x7f202c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2030a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c0cb7c0 0x7f202c1649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2030a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c0ce790 con 0x7f202c164f20 2024-09-13T16:50:26.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2030a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c0b2c70 con 0x7f202c0cde70 2024-09-13T16:50:26.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2030a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1653b0 con 0x7f202c0cb7c0 2024-09-13T16:50:26.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f202b200700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f202c164f20 0x7f202c161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42132/0 (socket says 172.21.15.3:42132) 2024-09-13T16:50:26.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c0cb7c0 0x7f202c1649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f202b200700 1 -- 172.21.15.3:0/3434092644 learned_addr learned my addr 172.21.15.3:0/3434092644 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:26.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f202a800700 1 --2- 172.21.15.3:0/3434092644 >> v2:172.21.15.157:3300/0 conn(0x7f202c0cde70 0x7f202c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029e00700 1 -- 172.21.15.3:0/3434092644 >> v2:172.21.15.157:3300/0 conn(0x7f202c0cde70 msgr2=0x7f202c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029e00700 1 --2- 172.21.15.3:0/3434092644 >> v2:172.21.15.157:3300/0 conn(0x7f202c0cde70 0x7f202c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029e00700 1 -- 172.21.15.3:0/3434092644 >> v1:172.21.15.3:6789/0 conn(0x7f202c164f20 legacy=0x7f202c161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029e00700 1 -- 172.21.15.3:0/3434092644 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c0cc050 con 0x7f202c0cb7c0 2024-09-13T16:50:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029e00700 1 --2- 172.21.15.3:0/3434092644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c0cb7c0 0x7f202c1649e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2018009850 tx=0x7f2018009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9be9c4b2313c9e92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029400700 1 -- 172.21.15.3:0/3434092644 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018011830 con 0x7f202c0cb7c0 2024-09-13T16:50:26.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029400700 1 -- 172.21.15.3:0/3434092644 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2018011990 con 0x7f202c0cb7c0 2024-09-13T16:50:26.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.405+0000 7f2029400700 1 -- 172.21.15.3:0/3434092644 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018011d40 con 0x7f202c0cb7c0 2024-09-13T16:50:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 -- 172.21.15.3:0/3434092644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c0cb7c0 msgr2=0x7f202c1649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 --2- 172.21.15.3:0/3434092644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c0cb7c0 0x7f202c1649e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2018009850 tx=0x7f2018009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 -- 172.21.15.3:0/3434092644 shutdown_connections 2024-09-13T16:50:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 --2- 172.21.15.3:0/3434092644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c0cb7c0 0x7f202c1649e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 --2- 172.21.15.3:0/3434092644 >> v2:172.21.15.157:3300/0 conn(0x7f202c0cde70 0x7f202c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 -- 172.21.15.3:0/3434092644 >> 172.21.15.3:0/3434092644 conn(0x7f202c0af620 msgr2=0x7f202c0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 -- 172.21.15.3:0/3434092644 shutdown_connections 2024-09-13T16:50:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 -- 172.21.15.3:0/3434092644 wait complete. 2024-09-13T16:50:26.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 Processor -- start 2024-09-13T16:50:26.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.409+0000 7f2030a00700 1 -- start start 2024-09-13T16:50:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2030a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0cb7c0 0x7f202c150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2030a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202c0cde70 0x7f202c154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2030a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c164f20 0x7f202c156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2030a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c156850 con 0x7f202c0cb7c0 2024-09-13T16:50:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2030a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1524d0 con 0x7f202c0cde70 2024-09-13T16:50:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2030a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c155d60 con 0x7f202c164f20 2024-09-13T16:50:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0cb7c0 0x7f202c150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0cb7c0 0x7f202c150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41158/0 (socket says 172.21.15.3:41158) 2024-09-13T16:50:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 -- 172.21.15.3:0/870922728 learned_addr learned my addr 172.21.15.3:0/870922728 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 -- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c164f20 msgr2=0x7f202c156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202b200700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c164f20 0x7f202c156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2029e00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202c0cde70 0x7f202c154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c164f20 0x7f202c156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 -- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202c0cde70 msgr2=0x7f202c154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202c0cde70 0x7f202c154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 -- 172.21.15.3:0/870922728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180094b0 con 0x7f202c0cb7c0 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202b200700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c164f20 0x7f202c156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2029e00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202c0cde70 0x7f202c154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f202a800700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0cb7c0 0x7f202c150cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f201c00ea00 tx=0x7f201c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2022a00700 1 -- 172.21.15.3:0/870922728 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c009720 con 0x7f202c0cb7c0 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2022a00700 1 -- 172.21.15.3:0/870922728 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c00c850 con 0x7f202c0cb7c0 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c151210 con 0x7f202c0cb7c0 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2022a00700 1 -- 172.21.15.3:0/870922728 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00cbc0 con 0x7f202c0cb7c0 2024-09-13T16:50:26.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.413+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c159740 con 0x7f202c0cb7c0 2024-09-13T16:50:26.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.417+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c1649e0 con 0x7f202c0cb7c0 2024-09-13T16:50:26.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.417+0000 7f2022a00700 1 -- 172.21.15.3:0/870922728 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f201c00cd20 con 0x7f202c0cb7c0 2024-09-13T16:50:26.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.417+0000 7f2022a00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f200c066730 0x7f200c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.417+0000 7f2022a00700 1 -- 172.21.15.3:0/870922728 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f201c084060 con 0x7f202c0cb7c0 2024-09-13T16:50:26.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.417+0000 7f2029e00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f200c066730 0x7f200c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.421+0000 7f2029e00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f200c066730 0x7f200c068bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f202c06cd40 tx=0x7f2018008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.425+0000 7f2022a00700 1 -- 172.21.15.3:0/870922728 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f201c0b5db0 con 0x7f202c0cb7c0 2024-09-13T16:50:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.601+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f202c06b5e0 con 0x7f202c0cb7c0 2024-09-13T16:50:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.605+0000 7f2022a00700 1 -- 172.21.15.3:0/870922728 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f201c055120 con 0x7f202c0cb7c0 2024-09-13T16:50:26.608 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:50:26.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f200c066730 msgr2=0x7f200c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f200c066730 0x7f200c068bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f202c06cd40 tx=0x7f2018008030 comp rx=0 tx=0).stop 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0cb7c0 msgr2=0x7f202c150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0cb7c0 0x7f202c150cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f201c00ea00 tx=0x7f201c00edc0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 shutdown_connections 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f200c066730 0x7f200c068bf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202c164f20 0x7f202c156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202c0cde70 0x7f202c154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 --2- 172.21.15.3:0/870922728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0cb7c0 0x7f202c150cd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 >> 172.21.15.3:0/870922728 conn(0x7f202c0af620 msgr2=0x7f202c06d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 shutdown_connections 2024-09-13T16:50:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.609+0000 7f2030a00700 1 -- 172.21.15.3:0/870922728 wait complete. 2024-09-13T16:50:26.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:26.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.809+0000 7f060ac00700 1 Processor -- start 2024-09-13T16:50:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.809+0000 7f060ac00700 1 -- start start 2024-09-13T16:50:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f060ac00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0604070db0 0x7f0604071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f060ac00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0604071750 con 0x7f060406c540 2024-09-13T16:50:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f060ac00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040b3450 con 0x7f060415b660 2024-09-13T16:50:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f060ac00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06040b40d0 con 0x7f0604070db0 2024-09-13T16:50:26.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f0608c00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f060406c540 0x7f060406c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42136/0 (socket says 172.21.15.3:42136) 2024-09-13T16:50:26.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f0608c00700 1 -- 172.21.15.3:0/2295882778 learned_addr learned my addr 172.21.15.3:0/2295882778 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:26.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f05fb400700 1 --2- 172.21.15.3:0/2295882778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0604070db0 0x7f0604071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f0603400700 1 -- 172.21.15.3:0/2295882778 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738200194 0 0) 0x7f0604071750 con 0x7f060406c540 2024-09-13T16:50:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.813+0000 7f0603400700 1 -- 172.21.15.3:0/2295882778 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e80035d0 con 0x7f060406c540 2024-09-13T16:50:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f0603400700 1 -- 172.21.15.3:0/2295882778 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903071429 0 0) 0x7f06040b3450 con 0x7f060415b660 2024-09-13T16:50:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f05fb400700 1 -- 172.21.15.3:0/2295882778 >> v1:172.21.15.157:6789/0 conn(0x7f060415b660 legacy=0x7f06040b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f05fb400700 1 -- 172.21.15.3:0/2295882778 >> v1:172.21.15.3:6789/0 conn(0x7f060406c540 legacy=0x7f060406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f05fb400700 1 -- 172.21.15.3:0/2295882778 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040b3730 con 0x7f0604070db0 2024-09-13T16:50:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f05fb400700 1 --2- 172.21.15.3:0/2295882778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0604070db0 0x7f0604071210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009880 tx=0x7f05ec009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e9818d10cae48d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f0603400700 1 -- 172.21.15.3:0/2295882778 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011830 con 0x7f0604070db0 2024-09-13T16:50:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f0603400700 1 -- 172.21.15.3:0/2295882778 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec011990 con 0x7f0604070db0 2024-09-13T16:50:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f0603400700 1 -- 172.21.15.3:0/2295882778 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011d20 con 0x7f0604070db0 2024-09-13T16:50:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f060ac00700 1 -- 172.21.15.3:0/2295882778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0604070db0 msgr2=0x7f0604071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f060ac00700 1 --2- 172.21.15.3:0/2295882778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0604070db0 0x7f0604071210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009880 tx=0x7f05ec009b90 comp rx=0 tx=0).stop 2024-09-13T16:50:26.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f060ac00700 1 -- 172.21.15.3:0/2295882778 shutdown_connections 2024-09-13T16:50:26.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f060ac00700 1 --2- 172.21.15.3:0/2295882778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0604070db0 0x7f0604071210 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f060ac00700 1 -- 172.21.15.3:0/2295882778 >> 172.21.15.3:0/2295882778 conn(0x7f060406a390 msgr2=0x7f060406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:26.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f060ac00700 1 -- 172.21.15.3:0/2295882778 shutdown_connections 2024-09-13T16:50:26.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.817+0000 7f060ac00700 1 -- 172.21.15.3:0/2295882778 wait complete. 2024-09-13T16:50:26.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f060ac00700 1 Processor -- start 2024-09-13T16:50:26.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f060ac00700 1 -- start start 2024-09-13T16:50:26.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f060ac00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060406c540 0x7f06041448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f060ac00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0604070db0 0x7f0604144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f060ac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060415b660 0x7f0604142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f060ac00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0604144500 con 0x7f0604070db0 2024-09-13T16:50:26.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f060ac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0604143880 con 0x7f060415b660 2024-09-13T16:50:26.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f060ac00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0604143b60 con 0x7f060406c540 2024-09-13T16:50:26.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f05fb400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0604070db0 0x7f0604144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f05fb400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0604070db0 0x7f0604144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41174/0 (socket says 172.21.15.3:41174) 2024-09-13T16:50:26.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f05fb400700 1 -- 172.21.15.3:0/3991394964 learned_addr learned my addr 172.21.15.3:0/3991394964 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:26.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f0608c00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060415b660 0x7f0604142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f0603e00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060406c540 0x7f06041448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f05fb400700 1 -- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060406c540 msgr2=0x7f06041448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f05fb400700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060406c540 0x7f06041448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f05fb400700 1 -- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060415b660 msgr2=0x7f0604142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f05fb400700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060415b660 0x7f0604142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.821+0000 7f05fb400700 1 -- 172.21.15.3:0/3991394964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0094b0 con 0x7f0604070db0 2024-09-13T16:50:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.825+0000 7f05fb400700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0604070db0 0x7f0604144e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05ec013040 tx=0x7f05ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.825+0000 7f0600c00700 1 -- 172.21.15.3:0/3991394964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec019950 con 0x7f0604070db0 2024-09-13T16:50:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.825+0000 7f0600c00700 1 -- 172.21.15.3:0/3991394964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec01e070 con 0x7f0604070db0 2024-09-13T16:50:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.825+0000 7f0600c00700 1 -- 172.21.15.3:0/3991394964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec019cd0 con 0x7f0604070db0 2024-09-13T16:50:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.825+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0604143e40 con 0x7f0604070db0 2024-09-13T16:50:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.825+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060415a190 con 0x7f0604070db0 2024-09-13T16:50:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.825+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060415ba40 con 0x7f0604070db0 2024-09-13T16:50:26.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.829+0000 7f0600c00700 1 -- 172.21.15.3:0/3991394964 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f05ec02d450 con 0x7f0604070db0 2024-09-13T16:50:26.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.829+0000 7f0600c00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05e0066730 0x7f05e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:26.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.829+0000 7f0600c00700 1 -- 172.21.15.3:0/3991394964 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05ec084dc0 con 0x7f0604070db0 2024-09-13T16:50:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.829+0000 7f0603e00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05e0066730 0x7f05e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.833+0000 7f0603e00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05e0066730 0x7f05e0068bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f05f4000f60 tx=0x7f05f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:26.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:26.837+0000 7f0600c00700 1 -- 172.21.15.3:0/3991394964 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f05ec052520 con 0x7f0604070db0 2024-09-13T16:50:27.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.061+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0604144640 con 0x7f0604070db0 2024-09-13T16:50:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.061+0000 7f0600c00700 1 -- 172.21.15.3:0/3991394964 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f05ec020090 con 0x7f0604070db0 2024-09-13T16:50:27.067 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:27.067 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T16:50:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05e0066730 msgr2=0x7f05e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05e0066730 0x7f05e0068bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f05f4000f60 tx=0x7f05f4009040 comp rx=0 tx=0).stop 2024-09-13T16:50:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0604070db0 msgr2=0x7f0604144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0604070db0 0x7f0604144e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05ec013040 tx=0x7f05ec004770 comp rx=0 tx=0).stop 2024-09-13T16:50:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 shutdown_connections 2024-09-13T16:50:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05e0066730 0x7f05e0068bf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060415b660 0x7f0604142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0604070db0 0x7f0604144e10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 --2- 172.21.15.3:0/3991394964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060406c540 0x7f06041448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 >> 172.21.15.3:0/3991394964 conn(0x7f060406a390 msgr2=0x7f06040ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 shutdown_connections 2024-09-13T16:50:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.065+0000 7f060ac00700 1 -- 172.21.15.3:0/3991394964 wait complete. 2024-09-13T16:50:27.073 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-09-13T16:50:27.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.257+0000 7fce76000700 1 Processor -- start 2024-09-13T16:50:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.257+0000 7fce76000700 1 -- start start 2024-09-13T16:50:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce76000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce701443b0 0x7fce700b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce76000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700ce8e0 con 0x7fce7006ccf0 2024-09-13T16:50:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce76000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700ccd30 con 0x7fce70144790 2024-09-13T16:50:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce76000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700cd010 con 0x7fce701443b0 2024-09-13T16:50:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce75600700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fce7006ccf0 0x7fce7015b220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42144/0 (socket says 172.21.15.3:42144) 2024-09-13T16:50:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce75600700 1 -- 172.21.15.3:0/3222393050 learned_addr learned my addr 172.21.15.3:0/3222393050 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce74c00700 1 --2- 172.21.15.3:0/3222393050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce701443b0 0x7fce700b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce6fe00700 1 -- 172.21.15.3:0/3222393050 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641481043 0 0) 0x7fce700ce8e0 con 0x7fce7006ccf0 2024-09-13T16:50:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce6fe00700 1 -- 172.21.15.3:0/3222393050 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce540035d0 con 0x7fce7006ccf0 2024-09-13T16:50:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce6fe00700 1 -- 172.21.15.3:0/3222393050 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765665222 0 0) 0x7fce700ccd30 con 0x7fce70144790 2024-09-13T16:50:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce6fe00700 1 -- 172.21.15.3:0/3222393050 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce700ce8e0 con 0x7fce70144790 2024-09-13T16:50:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce74c00700 1 -- 172.21.15.3:0/3222393050 >> v1:172.21.15.157:6789/0 conn(0x7fce70144790 legacy=0x7fce7006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce74c00700 1 -- 172.21.15.3:0/3222393050 >> v1:172.21.15.3:6789/0 conn(0x7fce7006ccf0 legacy=0x7fce7015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.261+0000 7fce74c00700 1 -- 172.21.15.3:0/3222393050 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700cd2f0 con 0x7fce701443b0 2024-09-13T16:50:27.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce74c00700 1 --2- 172.21.15.3:0/3222393050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce701443b0 0x7fce700b39f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fce60009880 tx=0x7fce60009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b3d7b679607d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:27.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce6fe00700 1 -- 172.21.15.3:0/3222393050 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60011830 con 0x7fce701443b0 2024-09-13T16:50:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce6fe00700 1 -- 172.21.15.3:0/3222393050 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60011990 con 0x7fce701443b0 2024-09-13T16:50:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce6fe00700 1 -- 172.21.15.3:0/3222393050 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60011d20 con 0x7fce701443b0 2024-09-13T16:50:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce76000700 1 -- 172.21.15.3:0/3222393050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce701443b0 msgr2=0x7fce700b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce76000700 1 --2- 172.21.15.3:0/3222393050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce701443b0 0x7fce700b39f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fce60009880 tx=0x7fce60009b90 comp rx=0 tx=0).stop 2024-09-13T16:50:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce76000700 1 -- 172.21.15.3:0/3222393050 shutdown_connections 2024-09-13T16:50:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce76000700 1 --2- 172.21.15.3:0/3222393050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce701443b0 0x7fce700b39f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce76000700 1 -- 172.21.15.3:0/3222393050 >> 172.21.15.3:0/3222393050 conn(0x7fce700b0cf0 msgr2=0x7fce700b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce76000700 1 -- 172.21.15.3:0/3222393050 shutdown_connections 2024-09-13T16:50:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.265+0000 7fce76000700 1 -- 172.21.15.3:0/3222393050 wait complete. 2024-09-13T16:50:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce76000700 1 Processor -- start 2024-09-13T16:50:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce76000700 1 -- start start 2024-09-13T16:50:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce76000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7006ccf0 0x7fce7013adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce76000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce701443b0 0x7fce7013f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce76000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7013a890 0x7fce700aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce76000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7013e8f0 con 0x7fce701443b0 2024-09-13T16:50:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce76000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7013ebd0 con 0x7fce7006ccf0 2024-09-13T16:50:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce76000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7013eeb0 con 0x7fce7013a890 2024-09-13T16:50:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce701443b0 0x7fce7013f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce701443b0 0x7fce7013f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41182/0 (socket says 172.21.15.3:41182) 2024-09-13T16:50:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 -- 172.21.15.3:0/3258741689 learned_addr learned my addr 172.21.15.3:0/3258741689 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce74c00700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7006ccf0 0x7fce7013adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce75600700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7013a890 0x7fce700aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 -- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7013a890 msgr2=0x7fce700aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7013a890 0x7fce700aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 -- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7006ccf0 msgr2=0x7fce7013adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7006ccf0 0x7fce7013adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 -- 172.21.15.3:0/3258741689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600094b0 con 0x7fce701443b0 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.269+0000 7fce67e00700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce701443b0 0x7fce7013f990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c490 tx=0x7fce5800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.273+0000 7fce6d600700 1 -- 172.21.15.3:0/3258741689 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580052d0 con 0x7fce701443b0 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.273+0000 7fce6d600700 1 -- 172.21.15.3:0/3258741689 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce58005450 con 0x7fce701443b0 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.273+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7013f190 con 0x7fce701443b0 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.273+0000 7fce6d600700 1 -- 172.21.15.3:0/3258741689 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580038e0 con 0x7fce701443b0 2024-09-13T16:50:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.273+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7013fed0 con 0x7fce701443b0 2024-09-13T16:50:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.273+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7013f6b0 con 0x7fce701443b0 2024-09-13T16:50:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.273+0000 7fce6d600700 1 -- 172.21.15.3:0/3258741689 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce58003a40 con 0x7fce701443b0 2024-09-13T16:50:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.277+0000 7fce6d600700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce4c066730 0x7fce4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.277+0000 7fce6d600700 1 -- 172.21.15.3:0/3258741689 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fce58083300 con 0x7fce701443b0 2024-09-13T16:50:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.277+0000 7fce74c00700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce4c066730 0x7fce4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.277+0000 7fce74c00700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce4c066730 0x7fce4c068bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce600090b0 tx=0x7fce6001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:27.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.285+0000 7fce6d600700 1 -- 172.21.15.3:0/3258741689 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce7013f6b0 con 0x7fce701443b0 2024-09-13T16:50:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.481+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce7013f6b0 con 0x7fce701443b0 2024-09-13T16:50:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.485+0000 7fce6d600700 1 -- 172.21.15.3:0/3258741689 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7fce7013f6b0 con 0x7fce701443b0 2024-09-13T16:50:27.489 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:27.489 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:21.998415+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.485+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce4c066730 msgr2=0x7fce4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.485+0000 7fce76000700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce4c066730 0x7fce4c068bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce600090b0 tx=0x7fce6001c040 comp rx=0 tx=0).stop 2024-09-13T16:50:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.485+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce701443b0 msgr2=0x7fce7013f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.485+0000 7fce76000700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce701443b0 0x7fce7013f990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c490 tx=0x7fce5800c7a0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.489+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 shutdown_connections 2024-09-13T16:50:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.489+0000 7fce76000700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce4c066730 0x7fce4c068bf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.489+0000 7fce76000700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7013a890 0x7fce700aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.489+0000 7fce76000700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce701443b0 0x7fce7013f990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.489+0000 7fce76000700 1 --2- 172.21.15.3:0/3258741689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7006ccf0 0x7fce7013adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.489+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 >> 172.21.15.3:0/3258741689 conn(0x7fce700b0cf0 msgr2=0x7fce701415e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.489+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 shutdown_connections 2024-09-13T16:50:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.489+0000 7fce76000700 1 -- 172.21.15.3:0/3258741689 wait complete. 2024-09-13T16:50:27.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:50:27.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:50:27.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.689+0000 7fb3ec000700 1 Processor -- start 2024-09-13T16:50:27.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.689+0000 7fb3ec000700 1 -- start start 2024-09-13T16:50:27.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ec000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e4070080 0x7fb3e40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ec000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e40b4d50 con 0x7fb3e415ab10 2024-09-13T16:50:27.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ec000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e40cfde0 con 0x7fb3e406f0e0 2024-09-13T16:50:27.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ec000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e406f550 con 0x7fb3e4070080 2024-09-13T16:50:27.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3e9600700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb3e415ab10 0x7fb3e40d0270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42152/0 (socket says 172.21.15.3:42152) 2024-09-13T16:50:27.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3e9600700 1 -- 172.21.15.3:0/2744811085 learned_addr learned my addr 172.21.15.3:0/2744811085 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:27.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ea000700 1 --2- 172.21.15.3:0/2744811085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e4070080 0x7fb3e40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3e3e00700 1 -- 172.21.15.3:0/2744811085 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923741971 0 0) 0x7fb3e40b4d50 con 0x7fb3e415ab10 2024-09-13T16:50:27.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3e3e00700 1 -- 172.21.15.3:0/2744811085 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3cc0035d0 con 0x7fb3e415ab10 2024-09-13T16:50:27.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3e3e00700 1 -- 172.21.15.3:0/2744811085 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618843983 0 0) 0x7fb3e40cfde0 con 0x7fb3e406f0e0 2024-09-13T16:50:27.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ea000700 1 -- 172.21.15.3:0/2744811085 >> v1:172.21.15.157:6789/0 conn(0x7fb3e406f0e0 legacy=0x7fb3e406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ea000700 1 -- 172.21.15.3:0/2744811085 >> v1:172.21.15.3:6789/0 conn(0x7fb3e415ab10 legacy=0x7fb3e40d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ea000700 1 -- 172.21.15.3:0/2744811085 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40b4f80 con 0x7fb3e4070080 2024-09-13T16:50:27.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3ea000700 1 --2- 172.21.15.3:0/2744811085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e4070080 0x7fb3e40b4810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009dc0 tx=0x7fb3dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506752e91d4c8b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:27.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3e3e00700 1 -- 172.21.15.3:0/2744811085 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00e070 con 0x7fb3e4070080 2024-09-13T16:50:27.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3e3e00700 1 -- 172.21.15.3:0/2744811085 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc00bb20 con 0x7fb3e4070080 2024-09-13T16:50:27.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.693+0000 7fb3e3e00700 1 -- 172.21.15.3:0/2744811085 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00bed0 con 0x7fb3e4070080 2024-09-13T16:50:27.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 -- 172.21.15.3:0/2744811085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e4070080 msgr2=0x7fb3e40b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 --2- 172.21.15.3:0/2744811085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e4070080 0x7fb3e40b4810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009dc0 tx=0x7fb3dc00b330 comp rx=0 tx=0).stop 2024-09-13T16:50:27.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 -- 172.21.15.3:0/2744811085 shutdown_connections 2024-09-13T16:50:27.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 --2- 172.21.15.3:0/2744811085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e4070080 0x7fb3e40b4810 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 -- 172.21.15.3:0/2744811085 >> 172.21.15.3:0/2744811085 conn(0x7fb3e40b2b00 msgr2=0x7fb3e40b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:27.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 -- 172.21.15.3:0/2744811085 shutdown_connections 2024-09-13T16:50:27.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 -- 172.21.15.3:0/2744811085 wait complete. 2024-09-13T16:50:27.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 Processor -- start 2024-09-13T16:50:27.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.697+0000 7fb3ec000700 1 -- start start 2024-09-13T16:50:27.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ec000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3e406f0e0 0x7fb3e41e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ec000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e4070080 0x7fb3e41f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ec000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e415ab10 0x7fb3e41e3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ec000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40afeb0 con 0x7fb3e4070080 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ec000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41e3480 con 0x7fb3e406f0e0 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ec000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41f7770 con 0x7fb3e415ab10 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e4070080 0x7fb3e41f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e4070080 0x7fb3e41f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41198/0 (socket says 172.21.15.3:41198) 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 -- 172.21.15.3:0/1958169880 learned_addr learned my addr 172.21.15.3:0/1958169880 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e9600700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3e406f0e0 0x7fb3e41e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ea000700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e415ab10 0x7fb3e41e3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 -- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e415ab10 msgr2=0x7fb3e41e3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e415ab10 0x7fb3e41e3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 -- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3e406f0e0 msgr2=0x7fb3e41e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3e406f0e0 0x7fb3e41e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 -- 172.21.15.3:0/1958169880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc009a70 con 0x7fb3e4070080 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e9600700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3e406f0e0 0x7fb3e41e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e8c00700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e4070080 0x7fb3e41f6e60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00011e0 tx=0x7fb3d000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e1600700 1 -- 172.21.15.3:0/1958169880 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00054e0 con 0x7fb3e4070080 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e1600700 1 -- 172.21.15.3:0/1958169880 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d00036a0 con 0x7fb3e4070080 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e41f7a50 con 0x7fb3e4070080 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3e1600700 1 -- 172.21.15.3:0/1958169880 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0003a50 con 0x7fb3e4070080 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.701+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e41e3720 con 0x7fb3e4070080 2024-09-13T16:50:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.705+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e40b4810 con 0x7fb3e4070080 2024-09-13T16:50:27.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.705+0000 7fb3e1600700 1 -- 172.21.15.3:0/1958169880 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb3d0003bb0 con 0x7fb3e4070080 2024-09-13T16:50:27.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.709+0000 7fb3e1600700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb3c4066730 0x7fb3c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:27.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.709+0000 7fb3e1600700 1 -- 172.21.15.3:0/1958169880 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb3d0083360 con 0x7fb3e4070080 2024-09-13T16:50:27.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.709+0000 7fb3e9600700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb3c4066730 0x7fb3c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:27.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.709+0000 7fb3e9600700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb3c4066730 0x7fb3c4068bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8005930 tx=0x7fb3d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:27.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.713+0000 7fb3e1600700 1 -- 172.21.15.3:0/1958169880 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb3d0050a10 con 0x7fb3e4070080 2024-09-13T16:50:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.901+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3e40b21e0 con 0x7fb3e4070080 2024-09-13T16:50:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.901+0000 7fb3e1600700 1 -- 172.21.15.3:0/1958169880 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3d00170a0 con 0x7fb3e4070080 2024-09-13T16:50:27.907 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb3c4066730 msgr2=0x7fb3c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb3c4066730 0x7fb3c4068bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8005930 tx=0x7fb3d8009040 comp rx=0 tx=0).stop 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e4070080 msgr2=0x7fb3e41f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e4070080 0x7fb3e41f6e60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00011e0 tx=0x7fb3d000cae0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 shutdown_connections 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb3c4066730 0x7fb3c4068bf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3e415ab10 0x7fb3e41e3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e4070080 0x7fb3e41f6e60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 --2- 172.21.15.3:0/1958169880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3e406f0e0 0x7fb3e41e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 >> 172.21.15.3:0/1958169880 conn(0x7fb3e40b2b00 msgr2=0x7fb3e40b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 shutdown_connections 2024-09-13T16:50:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:27.905+0000 7fb3ec000700 1 -- 172.21.15.3:0/1958169880 wait complete. 2024-09-13T16:50:27.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T16:50:28.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.097+0000 7f6c27200700 1 Processor -- start 2024-09-13T16:50:28.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c27200700 1 -- start start 2024-09-13T16:50:28.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c27200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c281443b0 0x7f6c280b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c27200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2806ccf0 0x7f6c2815b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c27200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c280ce8e0 con 0x7f6c28144790 2024-09-13T16:50:28.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c27200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c280ccd30 con 0x7f6c281443b0 2024-09-13T16:50:28.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c27200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280cd010 con 0x7f6c2806ccf0 2024-09-13T16:50:28.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c25e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c281443b0 0x7f6c280b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c25e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c281443b0 0x7f6c280b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:37178/0 (socket says 172.21.15.3:37178) 2024-09-13T16:50:28.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c25e00700 1 -- 172.21.15.3:0/2434701989 learned_addr learned my addr 172.21.15.3:0/2434701989 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:28.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c26800700 1 --2- 172.21.15.3:0/2434701989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2806ccf0 0x7f6c2815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c1fe00700 1 --1- 172.21.15.3:0/2434701989 >> v1:172.21.15.157:6789/0 conn(0x7f6c28144790 0x7f6c2806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:58960/0 (socket says 172.21.15.3:58960) 2024-09-13T16:50:28.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c25e00700 1 -- 172.21.15.3:0/2434701989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2806ccf0 msgr2=0x7f6c2815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:28.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c25e00700 1 --2- 172.21.15.3:0/2434701989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2806ccf0 0x7f6c2815b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c25e00700 1 -- 172.21.15.3:0/2434701989 >> v1:172.21.15.157:6789/0 conn(0x7f6c28144790 legacy=0x7f6c2806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.101+0000 7f6c25e00700 1 -- 172.21.15.3:0/2434701989 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280cd2f0 con 0x7f6c281443b0 2024-09-13T16:50:28.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c25e00700 1 --2- 172.21.15.3:0/2434701989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c281443b0 0x7f6c280b39f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009910 tx=0x7f6c18009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3715151135e0c75c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:28.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c25400700 1 -- 172.21.15.3:0/2434701989 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18011830 con 0x7f6c281443b0 2024-09-13T16:50:28.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c25400700 1 -- 172.21.15.3:0/2434701989 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c18011990 con 0x7f6c281443b0 2024-09-13T16:50:28.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c25400700 1 -- 172.21.15.3:0/2434701989 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18011d40 con 0x7f6c281443b0 2024-09-13T16:50:28.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c27200700 1 -- 172.21.15.3:0/2434701989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c281443b0 msgr2=0x7f6c280b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c27200700 1 --2- 172.21.15.3:0/2434701989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c281443b0 0x7f6c280b39f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009910 tx=0x7f6c18009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c27200700 1 -- 172.21.15.3:0/2434701989 shutdown_connections 2024-09-13T16:50:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c27200700 1 --2- 172.21.15.3:0/2434701989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2806ccf0 0x7f6c2815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c27200700 1 --2- 172.21.15.3:0/2434701989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c281443b0 0x7f6c280b39f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c27200700 1 -- 172.21.15.3:0/2434701989 >> 172.21.15.3:0/2434701989 conn(0x7f6c280b0cf0 msgr2=0x7f6c280b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c27200700 1 -- 172.21.15.3:0/2434701989 shutdown_connections 2024-09-13T16:50:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.105+0000 7f6c27200700 1 -- 172.21.15.3:0/2434701989 wait complete. 2024-09-13T16:50:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c27200700 1 Processor -- start 2024-09-13T16:50:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c27200700 1 -- start start 2024-09-13T16:50:28.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c27200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2806ccf0 0x7f6c280a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c27200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c281443b0 0x7f6c28142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c27200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c281f2900 0x7f6c281560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c27200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280aac70 con 0x7f6c281f2900 2024-09-13T16:50:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c27200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280a9d90 con 0x7f6c2806ccf0 2024-09-13T16:50:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c27200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280aa060 con 0x7f6c281443b0 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c281f2900 0x7f6c281560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c281f2900 0x7f6c281560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41212/0 (socket says 172.21.15.3:41212) 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 -- 172.21.15.3:0/673453692 learned_addr learned my addr 172.21.15.3:0/673453692 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c25e00700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2806ccf0 0x7f6c280a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c1fe00700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c281443b0 0x7f6c28142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 -- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c281443b0 msgr2=0x7f6c28142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c281443b0 0x7f6c28142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 -- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2806ccf0 msgr2=0x7f6c280a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2806ccf0 0x7f6c280a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 -- 172.21.15.3:0/673453692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18009570 con 0x7f6c281f2900 2024-09-13T16:50:28.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.109+0000 7f6c26800700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c281f2900 0x7f6c281560e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000efc0 tx=0x7f6c2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:28.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.113+0000 7f6c1e000700 1 -- 172.21.15.3:0/673453692 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20009ea0 con 0x7f6c281f2900 2024-09-13T16:50:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.113+0000 7f6c1e000700 1 -- 172.21.15.3:0/673453692 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000f040 con 0x7f6c281f2900 2024-09-13T16:50:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.113+0000 7f6c1e000700 1 -- 172.21.15.3:0/673453692 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000ced0 con 0x7f6c281f2900 2024-09-13T16:50:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.113+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c280aa340 con 0x7f6c281f2900 2024-09-13T16:50:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.113+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c280aa900 con 0x7f6c281f2900 2024-09-13T16:50:28.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.113+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c280aae20 con 0x7f6c281f2900 2024-09-13T16:50:28.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.117+0000 7f6c1e000700 1 -- 172.21.15.3:0/673453692 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c20014480 con 0x7f6c281f2900 2024-09-13T16:50:28.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.117+0000 7f6c1e000700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6c04066730 0x7f6c04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.117+0000 7f6c1e000700 1 -- 172.21.15.3:0/673453692 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6c20084980 con 0x7f6c281f2900 2024-09-13T16:50:28.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.117+0000 7f6c25e00700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6c04066730 0x7f6c04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.121+0000 7f6c25e00700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6c04066730 0x7f6c04068bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6c18000c00 tx=0x7f6c18008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:28.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.125+0000 7f6c1e000700 1 -- 172.21.15.3:0/673453692 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c200520e0 con 0x7f6c281f2900 2024-09-13T16:50:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.309+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6c280b39f0 con 0x7f6c281f2900 2024-09-13T16:50:28.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.309+0000 7f6c1e000700 1 -- 172.21.15.3:0/673453692 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6c20018020 con 0x7f6c281f2900 2024-09-13T16:50:28.313 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:28.313 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-13T16:50:28.313 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:50:28.313 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-13T16:50:28.314 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-13T16:50:28.314 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:50:28.314 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-13T16:50:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6c04066730 msgr2=0x7f6c04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6c04066730 0x7f6c04068bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6c18000c00 tx=0x7f6c18008030 comp rx=0 tx=0).stop 2024-09-13T16:50:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c281f2900 msgr2=0x7f6c281560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c281f2900 0x7f6c281560e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000efc0 tx=0x7f6c2000c3a0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 shutdown_connections 2024-09-13T16:50:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6c04066730 0x7f6c04068bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c281f2900 0x7f6c281560e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c281443b0 0x7f6c28142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 --2- 172.21.15.3:0/673453692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2806ccf0 0x7f6c280a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 >> 172.21.15.3:0/673453692 conn(0x7f6c280b0cf0 msgr2=0x7f6c28158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 shutdown_connections 2024-09-13T16:50:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.313+0000 7f6c27200700 1 -- 172.21.15.3:0/673453692 wait complete. 2024-09-13T16:50:28.338 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T16:50:28.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T16:50:28.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:50:28.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:50:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.517+0000 7fa3f6600700 1 Processor -- start 2024-09-13T16:50:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.517+0000 7fa3f6600700 1 -- start start 2024-09-13T16:50:28.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3f6600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f00cd4c0 0x7fa3f0164d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3f6600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f01656f0 con 0x7fa3f0165290 2024-09-13T16:50:28.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3f6600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f00b59d0 con 0x7fa3f015b990 2024-09-13T16:50:28.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3f6600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00cd8a0 con 0x7fa3f00cd4c0 2024-09-13T16:50:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3eea00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa3f0165290 0x7fa3f015b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42160/0 (socket says 172.21.15.3:42160) 2024-09-13T16:50:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3eea00700 1 -- 172.21.15.3:0/1929794632 learned_addr learned my addr 172.21.15.3:0/1929794632 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3ef400700 1 --2- 172.21.15.3:0/1929794632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f00cd4c0 0x7fa3f0164d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3ee000700 1 -- 172.21.15.3:0/1929794632 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710252867 0 0) 0x7fa3f01656f0 con 0x7fa3f0165290 2024-09-13T16:50:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3ee000700 1 -- 172.21.15.3:0/1929794632 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d40035d0 con 0x7fa3f0165290 2024-09-13T16:50:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3ef400700 1 -- 172.21.15.3:0/1929794632 >> v1:172.21.15.157:6789/0 conn(0x7fa3f015b990 legacy=0x7fa3f00b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3ef400700 1 -- 172.21.15.3:0/1929794632 >> v1:172.21.15.3:6789/0 conn(0x7fa3f0165290 legacy=0x7fa3f015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3ef400700 1 -- 172.21.15.3:0/1929794632 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00b56f0 con 0x7fa3f00cd4c0 2024-09-13T16:50:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.521+0000 7fa3ef400700 1 --2- 172.21.15.3:0/1929794632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f00cd4c0 0x7fa3f0164d50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009880 tx=0x7fa3e0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd8db600cf1cf582 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3ee000700 1 -- 172.21.15.3:0/1929794632 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0011830 con 0x7fa3f00cd4c0 2024-09-13T16:50:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3ee000700 1 -- 172.21.15.3:0/1929794632 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0011990 con 0x7fa3f00cd4c0 2024-09-13T16:50:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3ee000700 1 -- 172.21.15.3:0/1929794632 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0011d00 con 0x7fa3f00cd4c0 2024-09-13T16:50:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 -- 172.21.15.3:0/1929794632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f00cd4c0 msgr2=0x7fa3f0164d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 --2- 172.21.15.3:0/1929794632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f00cd4c0 0x7fa3f0164d50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009880 tx=0x7fa3e0009b90 comp rx=0 tx=0).stop 2024-09-13T16:50:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 -- 172.21.15.3:0/1929794632 shutdown_connections 2024-09-13T16:50:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 --2- 172.21.15.3:0/1929794632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f00cd4c0 0x7fa3f0164d50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 -- 172.21.15.3:0/1929794632 >> 172.21.15.3:0/1929794632 conn(0x7fa3f00b2c40 msgr2=0x7fa3f00b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 -- 172.21.15.3:0/1929794632 shutdown_connections 2024-09-13T16:50:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 -- 172.21.15.3:0/1929794632 wait complete. 2024-09-13T16:50:28.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 Processor -- start 2024-09-13T16:50:28.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.525+0000 7fa3f6600700 1 -- start start 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3f6600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f00cd4c0 0x7fa3f00cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3f6600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f015b990 0x7fa3f00ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3f6600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f0165290 0x7fa3f0143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3f6600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f015d330 con 0x7fa3f015b990 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3f6600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00cf710 con 0x7fa3f00cd4c0 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3f6600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f015c9d0 con 0x7fa3f0165290 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f015b990 0x7fa3f00ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f015b990 0x7fa3f00ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41214/0 (socket says 172.21.15.3:41214) 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 -- 172.21.15.3:0/2435467806 learned_addr learned my addr 172.21.15.3:0/2435467806 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3efe00700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f0165290 0x7fa3f0143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3ef400700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f00cd4c0 0x7fa3f00cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 -- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f0165290 msgr2=0x7fa3f0143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f0165290 0x7fa3f0143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 -- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f00cd4c0 msgr2=0x7fa3f00cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f00cd4c0 0x7fa3f00cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 -- 172.21.15.3:0/2435467806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00094b0 con 0x7fa3f015b990 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3eea00700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f015b990 0x7fa3f00ceec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005930 tx=0x7fa3dc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3e7400700 1 -- 172.21.15.3:0/2435467806 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc005d90 con 0x7fa3f015b990 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3e7400700 1 -- 172.21.15.3:0/2435467806 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc0070d0 con 0x7fa3f015b990 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3e7400700 1 -- 172.21.15.3:0/2435467806 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc007480 con 0x7fa3f015b990 2024-09-13T16:50:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f015cc70 con 0x7fa3f015b990 2024-09-13T16:50:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.529+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f00cf400 con 0x7fa3f015b990 2024-09-13T16:50:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.533+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f015d470 con 0x7fa3f015b990 2024-09-13T16:50:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.533+0000 7fa3e7400700 1 -- 172.21.15.3:0/2435467806 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3dc0075e0 con 0x7fa3f015b990 2024-09-13T16:50:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.533+0000 7fa3e7400700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3d0066730 0x7fa3d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.533+0000 7fa3e7400700 1 -- 172.21.15.3:0/2435467806 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa3dc083230 con 0x7fa3f015b990 2024-09-13T16:50:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.537+0000 7fa3ef400700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3d0066730 0x7fa3d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.537+0000 7fa3ef400700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3d0066730 0x7fa3d0068bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00090b0 tx=0x7fa3e0010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:28.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.541+0000 7fa3e7400700 1 -- 172.21.15.3:0/2435467806 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa3f015d470 con 0x7fa3f015b990 2024-09-13T16:50:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:28.725+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa3f015d470 con 0x7fa3f015b990 2024-09-13T16:50:29.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.065+0000 7fa3e7400700 1 -- 172.21.15.3:0/2435467806 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa3f015d470 con 0x7fa3f015b990 2024-09-13T16:50:29.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3d0066730 msgr2=0x7fa3d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:29.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3d0066730 0x7fa3d0068bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00090b0 tx=0x7fa3e0010000 comp rx=0 tx=0).stop 2024-09-13T16:50:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f015b990 msgr2=0x7fa3f00ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f015b990 0x7fa3f00ceec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005930 tx=0x7fa3dc00c7b0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 shutdown_connections 2024-09-13T16:50:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3d0066730 0x7fa3d0068bf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3f0165290 0x7fa3f0143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f015b990 0x7fa3f00ceec0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 --2- 172.21.15.3:0/2435467806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f00cd4c0 0x7fa3f00cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 >> 172.21.15.3:0/2435467806 conn(0x7fa3f00b2c40 msgr2=0x7fa3f015ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:29.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.069+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 shutdown_connections 2024-09-13T16:50:29.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.073+0000 7fa3f6600700 1 -- 172.21.15.3:0/2435467806 wait complete. 2024-09-13T16:50:29.083 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-13T16:50:29.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-13T16:50:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6ba600700 1 Processor -- start 2024-09-13T16:50:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6ba600700 1 -- start start 2024-09-13T16:50:29.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6ba600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b415aa20 0x7fe6b415e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:29.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6ba600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6b406de80 0x7fe6b40b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:29.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6ba600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6b406a390 0x7fe6b406a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:29.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6ba600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b406e2c0 con 0x7fe6b406a390 2024-09-13T16:50:29.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6ba600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b406ad40 con 0x7fe6b406de80 2024-09-13T16:50:29.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6ba600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b415ae00 con 0x7fe6b415aa20 2024-09-13T16:50:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.289+0000 7fe6b3e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6b406a390 0x7fe6b406a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6b406de80 0x7fe6b40b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b3e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6b406a390 0x7fe6b406a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41222/0 (socket says 172.21.15.3:41222) 2024-09-13T16:50:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b3e00700 1 -- 172.21.15.3:0/2002715234 learned_addr learned my addr 172.21.15.3:0/2002715234 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b3400700 1 --2- 172.21.15.3:0/2002715234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b415aa20 0x7fe6b415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2a00700 1 -- 172.21.15.3:0/2002715234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b415aa20 msgr2=0x7fe6b415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2a00700 1 --2- 172.21.15.3:0/2002715234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b415aa20 0x7fe6b415e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2a00700 1 -- 172.21.15.3:0/2002715234 >> v2:172.21.15.3:3300/0 conn(0x7fe6b406a390 msgr2=0x7fe6b406a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2a00700 1 --2- 172.21.15.3:0/2002715234 >> v2:172.21.15.3:3300/0 conn(0x7fe6b406a390 0x7fe6b406a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2a00700 1 -- 172.21.15.3:0/2002715234 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40b5330 con 0x7fe6b406de80 2024-09-13T16:50:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b3e00700 1 --2- 172.21.15.3:0/2002715234 >> v2:172.21.15.3:3300/0 conn(0x7fe6b406a390 0x7fe6b406a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2a00700 1 --2- 172.21.15.3:0/2002715234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406de80 0x7fe6b40b4b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40670c0 tx=0x7fe6a0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2714d254f3a6f6f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2000700 1 -- 172.21.15.3:0/2002715234 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0011830 con 0x7fe6b406de80 2024-09-13T16:50:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2000700 1 -- 172.21.15.3:0/2002715234 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a0011990 con 0x7fe6b406de80 2024-09-13T16:50:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6b2000700 1 -- 172.21.15.3:0/2002715234 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0011d20 con 0x7fe6b406de80 2024-09-13T16:50:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 -- 172.21.15.3:0/2002715234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406de80 msgr2=0x7fe6b40b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 --2- 172.21.15.3:0/2002715234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406de80 0x7fe6b40b4b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40670c0 tx=0x7fe6a0009b30 comp rx=0 tx=0).stop 2024-09-13T16:50:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 -- 172.21.15.3:0/2002715234 shutdown_connections 2024-09-13T16:50:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 --2- 172.21.15.3:0/2002715234 >> v2:172.21.15.3:3300/0 conn(0x7fe6b406a390 0x7fe6b406a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 --2- 172.21.15.3:0/2002715234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406de80 0x7fe6b40b4b00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 --2- 172.21.15.3:0/2002715234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b415aa20 0x7fe6b415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 -- 172.21.15.3:0/2002715234 >> 172.21.15.3:0/2002715234 conn(0x7fe6b4071f10 msgr2=0x7fe6b406b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 -- 172.21.15.3:0/2002715234 shutdown_connections 2024-09-13T16:50:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.293+0000 7fe6ba600700 1 -- 172.21.15.3:0/2002715234 wait complete. 2024-09-13T16:50:29.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.297+0000 7fe6ba600700 1 Processor -- start 2024-09-13T16:50:29.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.297+0000 7fe6ba600700 1 -- start start 2024-09-13T16:50:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.297+0000 7fe6ba600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406a390 0x7fe6b4150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.297+0000 7fe6ba600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b406de80 0x7fe6b4155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.297+0000 7fe6ba600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b415aa20 0x7fe6b41f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.297+0000 7fe6ba600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4156530 con 0x7fe6b415aa20 2024-09-13T16:50:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.297+0000 7fe6ba600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4154160 con 0x7fe6b406a390 2024-09-13T16:50:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.297+0000 7fe6ba600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4154440 con 0x7fe6b406de80 2024-09-13T16:50:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b415aa20 0x7fe6b41f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b415aa20 0x7fe6b41f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41232/0 (socket says 172.21.15.3:41232) 2024-09-13T16:50:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 -- 172.21.15.3:0/459075454 learned_addr learned my addr 172.21.15.3:0/459075454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b2a00700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b406de80 0x7fe6b4155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3400700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406a390 0x7fe6b4150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 -- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b406de80 msgr2=0x7fe6b4155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b406de80 0x7fe6b4155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 -- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406a390 msgr2=0x7fe6b4150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406a390 0x7fe6b4150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 -- 172.21.15.3:0/459075454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac009b30 con 0x7fe6b415aa20 2024-09-13T16:50:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6b3e00700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b415aa20 0x7fe6b41f2780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00eed0 tx=0x7fe6ac00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6ab400700 1 -- 172.21.15.3:0/459075454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac018070 con 0x7fe6b415aa20 2024-09-13T16:50:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6ab400700 1 -- 172.21.15.3:0/459075454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac0092e0 con 0x7fe6b415aa20 2024-09-13T16:50:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a00094b0 con 0x7fe6b415aa20 2024-09-13T16:50:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6ab400700 1 -- 172.21.15.3:0/459075454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac010040 con 0x7fe6b415aa20 2024-09-13T16:50:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b4154a00 con 0x7fe6b415aa20 2024-09-13T16:50:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.301+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b406a800 con 0x7fe6b415aa20 2024-09-13T16:50:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.305+0000 7fe6ab400700 1 -- 172.21.15.3:0/459075454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6ac00b870 con 0x7fe6b415aa20 2024-09-13T16:50:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.305+0000 7fe6ab400700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe694066730 0x7fe694068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.305+0000 7fe6ab400700 1 -- 172.21.15.3:0/459075454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe6ac083ca0 con 0x7fe6b415aa20 2024-09-13T16:50:29.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.309+0000 7fe6b3400700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe694066730 0x7fe694068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:29.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.309+0000 7fe6b3400700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe694066730 0x7fe694068bf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009bf0 tx=0x7fe6a4009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.313+0000 7fe6ab400700 1 -- 172.21.15.3:0/459075454 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6ac051160 con 0x7fe6b415aa20 2024-09-13T16:50:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:29.497+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fe6b406e480 con 0x7fe6b415aa20 2024-09-13T16:50:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.081+0000 7fe6ab400700 1 -- 172.21.15.3:0/459075454 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fe6ac054780 con 0x7fe6b415aa20 2024-09-13T16:50:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe694066730 msgr2=0x7fe694068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe694066730 0x7fe694068bf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009bf0 tx=0x7fe6a4009240 comp rx=0 tx=0).stop 2024-09-13T16:50:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b415aa20 msgr2=0x7fe6b41f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b415aa20 0x7fe6b41f2780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00eed0 tx=0x7fe6ac00b390 comp rx=0 tx=0).stop 2024-09-13T16:50:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 shutdown_connections 2024-09-13T16:50:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe694066730 0x7fe694068bf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b415aa20 0x7fe6b41f2780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b406de80 0x7fe6b4155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 --2- 172.21.15.3:0/459075454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b406a390 0x7fe6b4150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 >> 172.21.15.3:0/459075454 conn(0x7fe6b4071f10 msgr2=0x7fe6b415b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 shutdown_connections 2024-09-13T16:50:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.093+0000 7fe6ba600700 1 -- 172.21.15.3:0/459075454 wait complete. 2024-09-13T16:50:30.105 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-13T16:50:30.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T16:50:30.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.309+0000 7ff465200700 1 Processor -- start 2024-09-13T16:50:30.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.309+0000 7ff465200700 1 -- start start 2024-09-13T16:50:30.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff465200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff46015ab10 0x7ff46006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff465200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4600b4810 0x7ff4600d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff465200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4600cfde0 con 0x7ff46006f970 2024-09-13T16:50:30.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff465200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600b4f80 con 0x7ff4600b4810 2024-09-13T16:50:30.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff465200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600cf790 con 0x7ff46015ab10 2024-09-13T16:50:30.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff45d400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4600b4810 0x7ff4600d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff45d400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4600b4810 0x7ff4600d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41248/0 (socket says 172.21.15.3:41248) 2024-09-13T16:50:30.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff45d400700 1 -- 172.21.15.3:0/811630805 learned_addr learned my addr 172.21.15.3:0/811630805 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:30.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.313+0000 7ff45de00700 1 --2- 172.21.15.3:0/811630805 >> v2:172.21.15.157:3300/0 conn(0x7ff46015ab10 0x7ff46006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45d400700 1 -- 172.21.15.3:0/811630805 >> v2:172.21.15.157:3300/0 conn(0x7ff46015ab10 msgr2=0x7ff46006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:30.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45d400700 1 --2- 172.21.15.3:0/811630805 >> v2:172.21.15.157:3300/0 conn(0x7ff46015ab10 0x7ff46006f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45d400700 1 -- 172.21.15.3:0/811630805 >> v1:172.21.15.157:6789/0 conn(0x7ff46006f970 legacy=0x7ff46015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45d400700 1 -- 172.21.15.3:0/811630805 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600cfa70 con 0x7ff4600b4810 2024-09-13T16:50:30.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45ca00700 1 -- 172.21.15.3:0/811630805 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617923471 0 0) 0x7ff4600cfde0 con 0x7ff46006f970 2024-09-13T16:50:30.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45d400700 1 --2- 172.21.15.3:0/811630805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4600b4810 0x7ff4600d0270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4600704c0 tx=0x7ff44c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b09b722a987f1a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:30.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45ca00700 1 -- 172.21.15.3:0/811630805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c011830 con 0x7ff4600b4810 2024-09-13T16:50:30.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45ca00700 1 -- 172.21.15.3:0/811630805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c011990 con 0x7ff4600b4810 2024-09-13T16:50:30.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff45ca00700 1 -- 172.21.15.3:0/811630805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c011d20 con 0x7ff4600b4810 2024-09-13T16:50:30.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff465200700 1 -- 172.21.15.3:0/811630805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4600b4810 msgr2=0x7ff4600d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff465200700 1 --2- 172.21.15.3:0/811630805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4600b4810 0x7ff4600d0270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4600704c0 tx=0x7ff44c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff465200700 1 -- 172.21.15.3:0/811630805 shutdown_connections 2024-09-13T16:50:30.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff465200700 1 --2- 172.21.15.3:0/811630805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4600b4810 0x7ff4600d0270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff465200700 1 --2- 172.21.15.3:0/811630805 >> v2:172.21.15.157:3300/0 conn(0x7ff46015ab10 0x7ff46006f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff465200700 1 -- 172.21.15.3:0/811630805 >> 172.21.15.3:0/811630805 conn(0x7ff4600b2b00 msgr2=0x7ff4600b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:30.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff465200700 1 -- 172.21.15.3:0/811630805 shutdown_connections 2024-09-13T16:50:30.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.317+0000 7ff465200700 1 -- 172.21.15.3:0/811630805 wait complete. 2024-09-13T16:50:30.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff465200700 1 Processor -- start 2024-09-13T16:50:30.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff465200700 1 -- start start 2024-09-13T16:50:30.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff465200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff46006f970 0x7ff460156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff465200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4600b4810 0x7ff4600ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff465200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46015ab10 0x7ff4601e0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff465200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600aee10 con 0x7ff46015ab10 2024-09-13T16:50:30.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff465200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600ad460 con 0x7ff4600b4810 2024-09-13T16:50:30.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff465200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4601568a0 con 0x7ff46006f970 2024-09-13T16:50:30.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff45e800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46015ab10 0x7ff4601e0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff45e800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46015ab10 0x7ff4601e0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41264/0 (socket says 172.21.15.3:41264) 2024-09-13T16:50:30.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff45e800700 1 -- 172.21.15.3:0/2919259130 learned_addr learned my addr 172.21.15.3:0/2919259130 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:30.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff45de00700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff46006f970 0x7ff460156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.321+0000 7ff45d400700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4600b4810 0x7ff4600ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff45e800700 1 -- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff46006f970 msgr2=0x7ff460156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff45e800700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff46006f970 0x7ff460156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff45e800700 1 -- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4600b4810 msgr2=0x7ff4600ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff45e800700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4600b4810 0x7ff4600ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff45e800700 1 -- 172.21.15.3:0/2919259130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c009570 con 0x7ff46015ab10 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff45e800700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46015ab10 0x7ff4601e0cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff45800cc70 tx=0x7ff45800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff456000700 1 -- 172.21.15.3:0/2919259130 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458003e50 con 0x7ff46015ab10 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff456000700 1 -- 172.21.15.3:0/2919259130 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800a6f0 con 0x7ff46015ab10 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff456000700 1 -- 172.21.15.3:0/2919259130 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800aaa0 con 0x7ff46015ab10 2024-09-13T16:50:30.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4600ad700 con 0x7ff46015ab10 2024-09-13T16:50:30.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4600adcc0 con 0x7ff46015ab10 2024-09-13T16:50:30.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.325+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4600adeb0 con 0x7ff46015ab10 2024-09-13T16:50:30.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.329+0000 7ff456000700 1 -- 172.21.15.3:0/2919259130 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff45800ac00 con 0x7ff46015ab10 2024-09-13T16:50:30.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.329+0000 7ff456000700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff440066730 0x7ff440068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.329+0000 7ff456000700 1 -- 172.21.15.3:0/2919259130 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff458083bd0 con 0x7ff46015ab10 2024-09-13T16:50:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.333+0000 7ff45de00700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff440066730 0x7ff440068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.333+0000 7ff45de00700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff440066730 0x7ff440068bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff46006d940 tx=0x7ff450009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:30.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.337+0000 7ff456000700 1 -- 172.21.15.3:0/2919259130 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff458050f00 con 0x7ff46015ab10 2024-09-13T16:50:30.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.565+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff4600d0270 con 0x7ff46015ab10 2024-09-13T16:50:30.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.577+0000 7ff456000700 1 -- 172.21.15.3:0/2919259130 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff4580170c0 con 0x7ff46015ab10 2024-09-13T16:50:30.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.581+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff440066730 msgr2=0x7ff440068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.581+0000 7ff465200700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff440066730 0x7ff440068bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff46006d940 tx=0x7ff450009240 comp rx=0 tx=0).stop 2024-09-13T16:50:30.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.581+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46015ab10 msgr2=0x7ff4601e0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.581+0000 7ff465200700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46015ab10 0x7ff4601e0cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff45800cc70 tx=0x7ff45800cf80 comp rx=0 tx=0).stop 2024-09-13T16:50:30.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.585+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 shutdown_connections 2024-09-13T16:50:30.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.585+0000 7ff465200700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff440066730 0x7ff440068bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.585+0000 7ff465200700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46015ab10 0x7ff4601e0cf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.585+0000 7ff465200700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4600b4810 0x7ff4600ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.585+0000 7ff465200700 1 --2- 172.21.15.3:0/2919259130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff46006f970 0x7ff460156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.585+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 >> 172.21.15.3:0/2919259130 conn(0x7ff4600b2b00 msgr2=0x7ff4600b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.585+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 shutdown_connections 2024-09-13T16:50:30.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.589+0000 7ff465200700 1 -- 172.21.15.3:0/2919259130 wait complete. 2024-09-13T16:50:30.604 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T16:50:30.628 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-13T16:50:30.629 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-13T16:50:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.797+0000 7f5cdf200700 1 Processor -- start 2024-09-13T16:50:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.797+0000 7f5cdf200700 1 -- start start 2024-09-13T16:50:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cdf200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd8154ff0 0x7f5cd815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cdf200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5cd815b770 0x7f5cd80cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cdf200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd80cdec0 con 0x7f5cd80cd520 2024-09-13T16:50:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cdf200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd80d06d0 con 0x7f5cd815b770 2024-09-13T16:50:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cdf200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd815bc40 con 0x7f5cd8154ff0 2024-09-13T16:50:30.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cde800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5cd815b770 0x7f5cd80cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cde800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5cd815b770 0x7f5cd80cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41272/0 (socket says 172.21.15.3:41272) 2024-09-13T16:50:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cdde00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd8154ff0 0x7f5cd815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cde800700 1 -- 172.21.15.3:0/3892712085 learned_addr learned my addr 172.21.15.3:0/3892712085 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cde800700 1 -- 172.21.15.3:0/3892712085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd8154ff0 msgr2=0x7f5cd815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cde800700 1 --2- 172.21.15.3:0/3892712085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd8154ff0 0x7f5cd815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cde800700 1 -- 172.21.15.3:0/3892712085 >> v1:172.21.15.3:6789/0 conn(0x7f5cd80cd520 legacy=0x7f5cd815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cde800700 1 -- 172.21.15.3:0/3892712085 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd815bf20 con 0x7f5cd815b770 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.801+0000 7f5cde800700 1 --2- 172.21.15.3:0/3892712085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd815b770 0x7f5cd80cd980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009e50 tx=0x7f5cd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9789391785e3ee46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdca00700 1 -- 172.21.15.3:0/3892712085 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400e070 con 0x7f5cd815b770 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdca00700 1 -- 172.21.15.3:0/3892712085 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd400b910 con 0x7f5cd815b770 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdca00700 1 -- 172.21.15.3:0/3892712085 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400bcc0 con 0x7f5cd815b770 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdf200700 1 -- 172.21.15.3:0/3892712085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd815b770 msgr2=0x7f5cd80cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdf200700 1 --2- 172.21.15.3:0/3892712085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd815b770 0x7f5cd80cd980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009e50 tx=0x7f5cd400b330 comp rx=0 tx=0).stop 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdf200700 1 -- 172.21.15.3:0/3892712085 shutdown_connections 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdf200700 1 --2- 172.21.15.3:0/3892712085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd815b770 0x7f5cd80cd980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdf200700 1 --2- 172.21.15.3:0/3892712085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd8154ff0 0x7f5cd815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdf200700 1 -- 172.21.15.3:0/3892712085 >> 172.21.15.3:0/3892712085 conn(0x7f5cd80b6210 msgr2=0x7f5cd80b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdf200700 1 -- 172.21.15.3:0/3892712085 shutdown_connections 2024-09-13T16:50:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.805+0000 7f5cdf200700 1 -- 172.21.15.3:0/3892712085 wait complete. 2024-09-13T16:50:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.809+0000 7f5cdf200700 1 Processor -- start 2024-09-13T16:50:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.809+0000 7f5cdf200700 1 -- start start 2024-09-13T16:50:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.809+0000 7f5cdf200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cd80cd520 0x7f5cd8148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.809+0000 7f5cdf200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd8154ff0 0x7f5cd813a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.809+0000 7f5cdf200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd815b770 0x7f5cd81f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.809+0000 7f5cdf200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd813aa70 con 0x7f5cd8154ff0 2024-09-13T16:50:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.809+0000 7f5cdf200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8150a60 con 0x7f5cd80cd520 2024-09-13T16:50:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.809+0000 7f5cdf200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8150f90 con 0x7f5cd815b770 2024-09-13T16:50:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd8154ff0 0x7f5cd813a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd8154ff0 0x7f5cd813a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41282/0 (socket says 172.21.15.3:41282) 2024-09-13T16:50:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 -- 172.21.15.3:0/4052872941 learned_addr learned my addr 172.21.15.3:0/4052872941 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdde00700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cd80cd520 0x7f5cd8148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cde800700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd815b770 0x7f5cd81f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 -- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd815b770 msgr2=0x7f5cd81f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd815b770 0x7f5cd81f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 -- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cd80cd520 msgr2=0x7f5cd8148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cd80cd520 0x7f5cd8148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 -- 172.21.15.3:0/4052872941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd4009b30 con 0x7f5cd8154ff0 2024-09-13T16:50:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdd400700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd8154ff0 0x7f5cd813a530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800c500 tx=0x7f5cc800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cd2000700 1 -- 172.21.15.3:0/4052872941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc80052d0 con 0x7f5cd8154ff0 2024-09-13T16:50:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cd2000700 1 -- 172.21.15.3:0/4052872941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc8005430 con 0x7f5cd8154ff0 2024-09-13T16:50:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd8152430 con 0x7f5cd8154ff0 2024-09-13T16:50:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cd2000700 1 -- 172.21.15.3:0/4052872941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc80055a0 con 0x7f5cd8154ff0 2024-09-13T16:50:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.813+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd8151bd0 con 0x7f5cd8154ff0 2024-09-13T16:50:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.817+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd80cd980 con 0x7f5cd8154ff0 2024-09-13T16:50:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.817+0000 7f5cd2000700 1 -- 172.21.15.3:0/4052872941 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5cc80038e0 con 0x7f5cd8154ff0 2024-09-13T16:50:30.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.817+0000 7f5cd2000700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5cbc066730 0x7f5cbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:30.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.817+0000 7f5cd2000700 1 -- 172.21.15.3:0/4052872941 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cc8083510 con 0x7f5cd8154ff0 2024-09-13T16:50:30.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.821+0000 7f5cdde00700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5cbc066730 0x7f5cbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.821+0000 7f5cdde00700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5cbc066730 0x7f5cbc068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009bf0 tx=0x7f5ccc009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:30.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:30.825+0000 7f5cd2000700 1 -- 172.21.15.3:0/4052872941 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5cc8050870 con 0x7f5cd8154ff0 2024-09-13T16:50:31.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.021+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f5cd8152210 con 0x7f5cd8154ff0 2024-09-13T16:50:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.581+0000 7f5cd2000700 1 -- 172.21.15.3:0/4052872941 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5cc8053e90 con 0x7f5cd8154ff0 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5cbc066730 msgr2=0x7f5cbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5cbc066730 0x7f5cbc068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009bf0 tx=0x7f5ccc009240 comp rx=0 tx=0).stop 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd8154ff0 msgr2=0x7f5cd813a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd8154ff0 0x7f5cd813a530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800c500 tx=0x7f5cc800c810 comp rx=0 tx=0).stop 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 shutdown_connections 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5cbc066730 0x7f5cbc068bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cd815b770 0x7f5cd81f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cd8154ff0 0x7f5cd813a530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 --2- 172.21.15.3:0/4052872941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cd80cd520 0x7f5cd8148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 >> 172.21.15.3:0/4052872941 conn(0x7f5cd80b6210 msgr2=0x7f5cd815def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 shutdown_connections 2024-09-13T16:50:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.585+0000 7f5cdf200700 1 -- 172.21.15.3:0/4052872941 wait complete. 2024-09-13T16:50:31.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T16:50:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.801+0000 7f4400c00700 1 Processor -- start 2024-09-13T16:50:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.801+0000 7f4400c00700 1 -- start start 2024-09-13T16:50:31.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f4400c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43fc0ccb00 0x7f43fc06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:31.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f4400c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc06f1f0 con 0x7f43fc06ee10 2024-09-13T16:50:31.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f4400c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc0cd190 con 0x7f43fc151b60 2024-09-13T16:50:31.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f4400c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc0ccf70 con 0x7f43fc0ccb00 2024-09-13T16:50:31.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43fa800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f43fc06ee10 0x7f43fc0ad4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42174/0 (socket says 172.21.15.3:42174) 2024-09-13T16:50:31.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43fa800700 1 -- 172.21.15.3:0/2110722046 learned_addr learned my addr 172.21.15.3:0/2110722046 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:31.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43f9400700 1 --2- 172.21.15.3:0/2110722046 >> v2:172.21.15.157:3300/0 conn(0x7f43fc0ccb00 0x7f43fc06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:31.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43f8a00700 1 -- 172.21.15.3:0/2110722046 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441025390 0 0) 0x7f43fc06f1f0 con 0x7f43fc06ee10 2024-09-13T16:50:31.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43f8a00700 1 -- 172.21.15.3:0/2110722046 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e00035d0 con 0x7f43fc06ee10 2024-09-13T16:50:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43f9400700 1 -- 172.21.15.3:0/2110722046 >> v1:172.21.15.157:6789/0 conn(0x7f43fc151b60 legacy=0x7f43fc0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43f9400700 1 -- 172.21.15.3:0/2110722046 >> v1:172.21.15.3:6789/0 conn(0x7f43fc06ee10 legacy=0x7f43fc0ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43f9400700 1 -- 172.21.15.3:0/2110722046 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0cd470 con 0x7f43fc0ccb00 2024-09-13T16:50:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.805+0000 7f43f9400700 1 --2- 172.21.15.3:0/2110722046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc0ccb00 0x7f43fc06e8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009850 tx=0x7f43e8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b34499b4ec7ed265 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f43f8a00700 1 -- 172.21.15.3:0/2110722046 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8011830 con 0x7f43fc0ccb00 2024-09-13T16:50:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f43f8a00700 1 -- 172.21.15.3:0/2110722046 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e8011990 con 0x7f43fc0ccb00 2024-09-13T16:50:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f43f8a00700 1 -- 172.21.15.3:0/2110722046 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8011d40 con 0x7f43fc0ccb00 2024-09-13T16:50:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 -- 172.21.15.3:0/2110722046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc0ccb00 msgr2=0x7f43fc06e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 --2- 172.21.15.3:0/2110722046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc0ccb00 0x7f43fc06e8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009850 tx=0x7f43e8009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 -- 172.21.15.3:0/2110722046 shutdown_connections 2024-09-13T16:50:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 --2- 172.21.15.3:0/2110722046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc0ccb00 0x7f43fc06e8d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 -- 172.21.15.3:0/2110722046 >> 172.21.15.3:0/2110722046 conn(0x7f43fc070820 msgr2=0x7f43fc070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 -- 172.21.15.3:0/2110722046 shutdown_connections 2024-09-13T16:50:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 -- 172.21.15.3:0/2110722046 wait complete. 2024-09-13T16:50:31.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 Processor -- start 2024-09-13T16:50:31.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.809+0000 7f4400c00700 1 -- start start 2024-09-13T16:50:31.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f4400c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06ee10 0x7f43fc150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:31.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f4400c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43fc0ccb00 0x7f43fc15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:31.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f4400c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc151b60 0x7f43fc15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:31.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f4400c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0b2650 con 0x7f43fc151b60 2024-09-13T16:50:31.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f4400c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc151470 con 0x7f43fc06ee10 2024-09-13T16:50:31.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f4400c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1f31c0 con 0x7f43fc0ccb00 2024-09-13T16:50:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc151b60 0x7f43fc15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc151b60 0x7f43fc15cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41292/0 (socket says 172.21.15.3:41292) 2024-09-13T16:50:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 -- 172.21.15.3:0/2849382425 learned_addr learned my addr 172.21.15.3:0/2849382425 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43f9400700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43fc0ccb00 0x7f43fc15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43f9e00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06ee10 0x7f43fc150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 -- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43fc0ccb00 msgr2=0x7f43fc15c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43fc0ccb00 0x7f43fc15c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 -- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06ee10 msgr2=0x7f43fc150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06ee10 0x7f43fc150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 -- 172.21.15.3:0/2849382425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e80094b0 con 0x7f43fc151b60 2024-09-13T16:50:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43fa800700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc151b60 0x7f43fc15cec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43f400cb40 tx=0x7f43f400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43f2000700 1 -- 172.21.15.3:0/2849382425 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4007500 con 0x7f43fc151b60 2024-09-13T16:50:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43f2000700 1 -- 172.21.15.3:0/2849382425 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f4007660 con 0x7f43fc151b60 2024-09-13T16:50:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc0b28a0 con 0x7f43fc151b60 2024-09-13T16:50:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f43f2000700 1 -- 172.21.15.3:0/2849382425 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40079f0 con 0x7f43fc151b60 2024-09-13T16:50:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.813+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc0b2e60 con 0x7f43fc151b60 2024-09-13T16:50:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.817+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc0b3050 con 0x7f43fc151b60 2024-09-13T16:50:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.817+0000 7f43f2000700 1 -- 172.21.15.3:0/2849382425 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43f401b3b0 con 0x7f43fc151b60 2024-09-13T16:50:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.817+0000 7f43f2000700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f43dc066730 0x7f43dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.817+0000 7f43f2000700 1 -- 172.21.15.3:0/2849382425 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f43f4083bd0 con 0x7f43fc151b60 2024-09-13T16:50:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.821+0000 7f43f9e00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f43dc066730 0x7f43dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.821+0000 7f43f9e00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f43dc066730 0x7f43dc068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43fc070d90 tx=0x7f43ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:31.825+0000 7f43f2000700 1 -- 172.21.15.3:0/2849382425 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f43f4050ef0 con 0x7f43fc151b60 2024-09-13T16:50:31.942 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-09-13T16:50:31.936+0000 7fcf3bc00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:50:32.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:32.013+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f43fc06e8d0 con 0x7f43fc151b60 2024-09-13T16:50:33.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.593+0000 7f43f2000700 1 -- 172.21.15.3:0/2849382425 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f43f4054510 con 0x7f43fc151b60 2024-09-13T16:50:33.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f43dc066730 msgr2=0x7f43dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f43dc066730 0x7f43dc068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43fc070d90 tx=0x7f43ec009040 comp rx=0 tx=0).stop 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc151b60 msgr2=0x7f43fc15cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc151b60 0x7f43fc15cec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43f400cb40 tx=0x7f43f400cb70 comp rx=0 tx=0).stop 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 shutdown_connections 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f43dc066730 0x7f43dc068bf0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43fc070d90 tx=0x7f43ec009040 comp rx=0 tx=0).stop 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc151b60 0x7f43fc15cec0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43fc0ccb00 0x7f43fc15c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 --2- 172.21.15.3:0/2849382425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06ee10 0x7f43fc150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 >> 172.21.15.3:0/2849382425 conn(0x7f43fc070820 msgr2=0x7f43fc15a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:33.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 shutdown_connections 2024-09-13T16:50:33.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.601+0000 7f4400c00700 1 -- 172.21.15.3:0/2849382425 wait complete. 2024-09-13T16:50:33.619 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' created 2024-09-13T16:50:33.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-13T16:50:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.801+0000 7fd4ee400700 1 Processor -- start 2024-09-13T16:50:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.801+0000 7fd4ee400700 1 -- start start 2024-09-13T16:50:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4ee400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4e806eec0 0x7fd4e80cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4ee400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80cec20 con 0x7fd4e80ae860 2024-09-13T16:50:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4ee400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e8066a70 con 0x7fd4e815ad10 2024-09-13T16:50:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4ee400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e80b2570 con 0x7fd4e806eec0 2024-09-13T16:50:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e7e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd4e80ae860 0x7fd4e80b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48762/0 (socket says 172.21.15.3:48762) 2024-09-13T16:50:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e7e00700 1 -- 172.21.15.3:0/4266924321 learned_addr learned my addr 172.21.15.3:0/4266924321 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4dfe00700 1 --2- 172.21.15.3:0/4266924321 >> v2:172.21.15.157:3300/0 conn(0x7fd4e806eec0 0x7fd4e80cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112293927 0 0) 0x7fd4e80cec20 con 0x7fd4e80ae860 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4cc0035d0 con 0x7fd4e80ae860 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4064861869 0 0) 0x7fd4cc0035d0 con 0x7fd4e80ae860 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 >> v2:172.21.15.157:3300/0 conn(0x7fd4e806eec0 msgr2=0x7fd4e80cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 --2- 172.21.15.3:0/4266924321 >> v2:172.21.15.157:3300/0 conn(0x7fd4e806eec0 0x7fd4e80cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 >> v1:172.21.15.157:6789/0 conn(0x7fd4e815ad10 legacy=0x7fd4e806e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80ceea0 con 0x7fd4e80ae860 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fd4e0001a10 con 0x7fd4e80ae860 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 >> v1:172.21.15.3:6789/0 conn(0x7fd4e80ae860 legacy=0x7fd4e80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4cc003a20 0x7fd4cc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc004870 0x7fd4cc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc005230 0x7fd4cc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80b2570 con 0x7fd4cc004870 2024-09-13T16:50:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc00c1d0 con 0x7fd4cc005230 2024-09-13T16:50:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc00c390 con 0x7fd4cc003a20 2024-09-13T16:50:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4dfe00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc004870 0x7fd4cc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e7e00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc005230 0x7fd4cc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.805+0000 7fd4e7400700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4cc003a20 0x7fd4cc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4dfe00700 1 -- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4cc003a20 msgr2=0x7fd4cc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4dfe00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4cc003a20 0x7fd4cc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4dfe00700 1 -- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc005230 msgr2=0x7fd4cc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4dfe00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc005230 0x7fd4cc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4dfe00700 1 -- 172.21.15.3:0/4266924321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0035d0 con 0x7fd4cc004870 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4e7e00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc005230 0x7fd4cc00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4e7e00700 1 -- 172.21.15.3:0/4266924321 reap_dead start 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4dfe00700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc004870 0x7fd4cc004cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009b40 tx=0x7fd4cc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d950bb1ea11a635d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000e6c0 con 0x7fd4cc004870 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d000e820 con 0x7fd4cc004870 2024-09-13T16:50:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4e6a00700 1 -- 172.21.15.3:0/4266924321 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000eb90 con 0x7fd4cc004870 2024-09-13T16:50:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4ee400700 1 -- 172.21.15.3:0/4266924321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc004870 msgr2=0x7fd4cc004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4ee400700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc004870 0x7fd4cc004cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009b40 tx=0x7fd4cc0058a0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4ee400700 1 -- 172.21.15.3:0/4266924321 shutdown_connections 2024-09-13T16:50:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4ee400700 1 --2- 172.21.15.3:0/4266924321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc004870 0x7fd4cc004cf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4ee400700 1 -- 172.21.15.3:0/4266924321 >> 172.21.15.3:0/4266924321 conn(0x7fd4e80716c0 msgr2=0x7fd4e806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4ee400700 1 -- 172.21.15.3:0/4266924321 shutdown_connections 2024-09-13T16:50:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.809+0000 7fd4ee400700 1 -- 172.21.15.3:0/4266924321 wait complete. 2024-09-13T16:50:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4ee400700 1 Processor -- start 2024-09-13T16:50:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4ee400700 1 -- start start 2024-09-13T16:50:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4ee400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc004870 0x7fd4e80ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4ee400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4e81433e0 0x7fd4e813ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4ee400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4e80c74f0 0x7fd4e80c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4ee400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c7ec0 con 0x7fd4e81433e0 2024-09-13T16:50:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4ee400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8143860 con 0x7fd4cc004870 2024-09-13T16:50:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4ee400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80abc00 con 0x7fd4e80c74f0 2024-09-13T16:50:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4dfe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4e81433e0 0x7fd4e813ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4dfe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4e81433e0 0x7fd4e813ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44446/0 (socket says 172.21.15.3:44446) 2024-09-13T16:50:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4dfe00700 1 -- 172.21.15.3:0/561230316 learned_addr learned my addr 172.21.15.3:0/561230316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:33.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4e7400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc004870 0x7fd4e80ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4e7e00700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4e80c74f0 0x7fd4e80c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4dfe00700 1 -- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4e80c74f0 msgr2=0x7fd4e80c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4dfe00700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4e80c74f0 0x7fd4e80c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4dfe00700 1 -- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc004870 msgr2=0x7fd4e80ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4dfe00700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc004870 0x7fd4e80ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.813+0000 7fd4dfe00700 1 -- 172.21.15.3:0/561230316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80abee0 con 0x7fd4e81433e0 2024-09-13T16:50:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.817+0000 7fd4dfe00700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4e81433e0 0x7fd4e813ffd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c1a0 tx=0x7fd4d00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.817+0000 7fd4df400700 1 -- 172.21.15.3:0/561230316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000e6c0 con 0x7fd4e81433e0 2024-09-13T16:50:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.817+0000 7fd4df400700 1 -- 172.21.15.3:0/561230316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d0013e00 con 0x7fd4e81433e0 2024-09-13T16:50:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.817+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e80ac1c0 con 0x7fd4e81433e0 2024-09-13T16:50:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.817+0000 7fd4df400700 1 -- 172.21.15.3:0/561230316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d001c690 con 0x7fd4e81433e0 2024-09-13T16:50:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.817+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e80ac780 con 0x7fd4e81433e0 2024-09-13T16:50:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.817+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e80c7ec0 con 0x7fd4e81433e0 2024-09-13T16:50:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.821+0000 7fd4df400700 1 -- 172.21.15.3:0/561230316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4d001c7f0 con 0x7fd4e81433e0 2024-09-13T16:50:33.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.821+0000 7fd4df400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4c4066730 0x7fd4c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.821+0000 7fd4df400700 1 -- 172.21.15.3:0/561230316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fd4d00860b0 con 0x7fd4e81433e0 2024-09-13T16:50:33.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.821+0000 7fd4e7400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4c4066730 0x7fd4c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.821+0000 7fd4e7400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4c4066730 0x7fd4c4068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8007fd0 tx=0x7fd4d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:33.825+0000 7fd4df400700 1 -- 172.21.15.3:0/561230316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd4d00531e0 con 0x7fd4e81433e0 2024-09-13T16:50:34.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.009+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd4e80ac940 con 0x7fd4e81433e0 2024-09-13T16:50:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.601+0000 7fd4df400700 1 -- 172.21.15.3:0/561230316 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd4d0056800 con 0x7fd4e81433e0 2024-09-13T16:50:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.617+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4c4066730 msgr2=0x7fd4c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.617+0000 7fd4ee400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4c4066730 0x7fd4c4068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8007fd0 tx=0x7fd4d8009040 comp rx=0 tx=0).stop 2024-09-13T16:50:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.617+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4e81433e0 msgr2=0x7fd4e813ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.617+0000 7fd4ee400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4e81433e0 0x7fd4e813ffd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c1a0 tx=0x7fd4d00047e0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.621+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 shutdown_connections 2024-09-13T16:50:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.621+0000 7fd4ee400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4c4066730 0x7fd4c4068bf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.621+0000 7fd4ee400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4e80c74f0 0x7fd4e80c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.621+0000 7fd4ee400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4e81433e0 0x7fd4e813ffd0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c1a0 tx=0x7fd4d00047e0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.621+0000 7fd4ee400700 1 --2- 172.21.15.3:0/561230316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4cc004870 0x7fd4e80ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.621+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 >> 172.21.15.3:0/561230316 conn(0x7fd4e80716c0 msgr2=0x7fd4e806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:34.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.625+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 shutdown_connections 2024-09-13T16:50:34.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.625+0000 7fd4ee400700 1 -- 172.21.15.3:0/561230316 wait complete. 2024-09-13T16:50:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 4 allow_ec_overwrites to true 2024-09-13T16:50:34.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-13T16:50:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.825+0000 7f0c81c00700 1 Processor -- start 2024-09-13T16:50:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.829+0000 7f0c81c00700 1 -- start start 2024-09-13T16:50:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.829+0000 7f0c81c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b5570 0x7f0c7c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.829+0000 7f0c81c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c7c0b0780 0x7f0c7c15ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.829+0000 7f0c81c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7c0b5a30 con 0x7f0c7c1444f0 2024-09-13T16:50:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.829+0000 7f0c81c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7c0b5d10 con 0x7f0c7c0b0780 2024-09-13T16:50:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.829+0000 7f0c81c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0cd5b0 con 0x7f0c7c0b5570 2024-09-13T16:50:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.829+0000 7f0c7be00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0c7c1444f0 0x7f0c7c0d0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48776/0 (socket says 172.21.15.3:48776) 2024-09-13T16:50:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.829+0000 7f0c7be00700 1 -- 172.21.15.3:0/2507679036 learned_addr learned my addr 172.21.15.3:0/2507679036 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81200700 1 --2- 172.21.15.3:0/2507679036 >> v2:172.21.15.3:3300/0 conn(0x7f0c7c0b0780 0x7f0c7c15ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c7b400700 1 --2- 172.21.15.3:0/2507679036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b5570 0x7f0c7c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c7aa00700 1 -- 172.21.15.3:0/2507679036 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530308986 0 0) 0x7f0c7c0b5a30 con 0x7f0c7c1444f0 2024-09-13T16:50:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c7aa00700 1 -- 172.21.15.3:0/2507679036 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c600035d0 con 0x7f0c7c1444f0 2024-09-13T16:50:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81200700 1 -- 172.21.15.3:0/2507679036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b5570 msgr2=0x7f0c7c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81200700 1 --2- 172.21.15.3:0/2507679036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b5570 0x7f0c7c15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81200700 1 -- 172.21.15.3:0/2507679036 >> v1:172.21.15.3:6789/0 conn(0x7f0c7c1444f0 legacy=0x7f0c7c0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81200700 1 -- 172.21.15.3:0/2507679036 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0cd890 con 0x7f0c7c0b0780 2024-09-13T16:50:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81200700 1 --2- 172.21.15.3:0/2507679036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c0b0780 0x7f0c7c15ebc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009d90 tx=0x7f0c7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1bcdb3264e67f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c7aa00700 1 -- 172.21.15.3:0/2507679036 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400e070 con 0x7f0c7c0b0780 2024-09-13T16:50:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c7aa00700 1 -- 172.21.15.3:0/2507679036 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7400bb20 con 0x7f0c7c0b0780 2024-09-13T16:50:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c7aa00700 1 -- 172.21.15.3:0/2507679036 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74012430 con 0x7f0c7c0b0780 2024-09-13T16:50:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81c00700 1 -- 172.21.15.3:0/2507679036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c0b0780 msgr2=0x7f0c7c15ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81c00700 1 --2- 172.21.15.3:0/2507679036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c0b0780 0x7f0c7c15ebc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009d90 tx=0x7f0c7400b330 comp rx=0 tx=0).stop 2024-09-13T16:50:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81c00700 1 -- 172.21.15.3:0/2507679036 shutdown_connections 2024-09-13T16:50:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81c00700 1 --2- 172.21.15.3:0/2507679036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c0b0780 0x7f0c7c15ebc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81c00700 1 --2- 172.21.15.3:0/2507679036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b5570 0x7f0c7c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81c00700 1 -- 172.21.15.3:0/2507679036 >> 172.21.15.3:0/2507679036 conn(0x7f0c7c0721a0 msgr2=0x7f0c7c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81c00700 1 -- 172.21.15.3:0/2507679036 shutdown_connections 2024-09-13T16:50:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.833+0000 7f0c81c00700 1 -- 172.21.15.3:0/2507679036 wait complete. 2024-09-13T16:50:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81c00700 1 Processor -- start 2024-09-13T16:50:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81c00700 1 -- start start 2024-09-13T16:50:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b0780 0x7f0c7c141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7c0b5570 0x7f0c7c158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c1444f0 0x7f0c7c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1598a0 con 0x7f0c7c1444f0 2024-09-13T16:50:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c158c20 con 0x7f0c7c0b5570 2024-09-13T16:50:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c158f00 con 0x7f0c7c0b0780 2024-09-13T16:50:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c1444f0 0x7f0c7c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c1444f0 0x7f0c7c1585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44456/0 (socket says 172.21.15.3:44456) 2024-09-13T16:50:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c81200700 1 -- 172.21.15.3:0/3584736639 learned_addr learned my addr 172.21.15.3:0/3584736639 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c7be00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b0780 0x7f0c7c141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.837+0000 7f0c7b400700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7c0b5570 0x7f0c7c158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81200700 1 -- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b0780 msgr2=0x7f0c7c141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81200700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b0780 0x7f0c7c141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81200700 1 -- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7c0b5570 msgr2=0x7f0c7c158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81200700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7c0b5570 0x7f0c7c158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81200700 1 -- 172.21.15.3:0/3584736639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c74009a70 con 0x7f0c7c1444f0 2024-09-13T16:50:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81200700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c1444f0 0x7f0c7c1585c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c74014040 tx=0x7f0c74004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:34.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c73e00700 1 -- 172.21.15.3:0/3584736639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74012430 con 0x7f0c7c1444f0 2024-09-13T16:50:34.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c73e00700 1 -- 172.21.15.3:0/3584736639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74020070 con 0x7f0c7c1444f0 2024-09-13T16:50:34.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c1591e0 con 0x7f0c7c1444f0 2024-09-13T16:50:34.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c73e00700 1 -- 172.21.15.3:0/3584736639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c740098f0 con 0x7f0c7c1444f0 2024-09-13T16:50:34.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c1f26d0 con 0x7f0c7c1444f0 2024-09-13T16:50:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.841+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7c0d0140 con 0x7f0c7c1444f0 2024-09-13T16:50:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.845+0000 7f0c73e00700 1 -- 172.21.15.3:0/3584736639 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c74004040 con 0x7f0c7c1444f0 2024-09-13T16:50:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.845+0000 7f0c73e00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0c5c066760 0x7f0c5c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.845+0000 7f0c73e00700 1 -- 172.21.15.3:0/3584736639 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f0c74022070 con 0x7f0c7c1444f0 2024-09-13T16:50:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.845+0000 7f0c7be00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0c5c066760 0x7f0c5c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.849+0000 7f0c7be00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0c5c066760 0x7f0c5c068c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c007fd0 tx=0x7f0c6c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:34.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:34.849+0000 7f0c73e00700 1 -- 172.21.15.3:0/3584736639 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0c7408a070 con 0x7f0c7c1444f0 2024-09-13T16:50:35.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.073+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f0c7c0cf270 con 0x7f0c7c1444f0 2024-09-13T16:50:35.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.085+0000 7f0c73e00700 1 -- 172.21.15.3:0/3584736639 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f0c7402c020 con 0x7f0c7c1444f0 2024-09-13T16:50:35.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0c5c066760 msgr2=0x7f0c5c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0c5c066760 0x7f0c5c068c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c007fd0 tx=0x7f0c6c007de0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c1444f0 msgr2=0x7f0c7c1585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c1444f0 0x7f0c7c1585c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c74014040 tx=0x7f0c74004af0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 shutdown_connections 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0c5c066760 0x7f0c5c068c20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7c1444f0 0x7f0c7c1585c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7c0b5570 0x7f0c7c158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 --2- 172.21.15.3:0/3584736639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c7c0b0780 0x7f0c7c141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 >> 172.21.15.3:0/3584736639 conn(0x7f0c7c0721a0 msgr2=0x7f0c7c0b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 shutdown_connections 2024-09-13T16:50:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.089+0000 7f0c81c00700 1 -- 172.21.15.3:0/3584736639 wait complete. 2024-09-13T16:50:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:added data pool 4 to fsmap 2024-09-13T16:50:35.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.281+0000 7fd09b600700 1 Processor -- start 2024-09-13T16:50:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.281+0000 7fd09b600700 1 -- start start 2024-09-13T16:50:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd09b600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd09415b660 0x7fd0940b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd09b600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940714c0 0x7fd09406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd09b600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940b3450 con 0x7fd0940b2f40 2024-09-13T16:50:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd09b600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940b40d0 con 0x7fd09415b660 2024-09-13T16:50:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd09b600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940b3730 con 0x7fd0940714c0 2024-09-13T16:50:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd098c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd09415b660 0x7fd0940b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd093e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd0940b2f40 0x7fd094070db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:50076/0 (socket says 172.21.15.3:50076) 2024-09-13T16:50:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd093e00700 1 -- 172.21.15.3:0/853016577 learned_addr learned my addr 172.21.15.3:0/853016577 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd099600700 1 --2- 172.21.15.3:0/853016577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940714c0 0x7fd09406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd093400700 1 -- 172.21.15.3:0/853016577 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291259288 0 0) 0x7fd0940b3450 con 0x7fd0940b2f40 2024-09-13T16:50:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.285+0000 7fd093400700 1 -- 172.21.15.3:0/853016577 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd07c0035d0 con 0x7fd0940b2f40 2024-09-13T16:50:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd098c00700 1 -- 172.21.15.3:0/853016577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940714c0 msgr2=0x7fd09406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd098c00700 1 --2- 172.21.15.3:0/853016577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940714c0 0x7fd09406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd098c00700 1 -- 172.21.15.3:0/853016577 >> v1:172.21.15.157:6789/0 conn(0x7fd0940b2f40 legacy=0x7fd094070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd098c00700 1 -- 172.21.15.3:0/853016577 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940b3a10 con 0x7fd09415b660 2024-09-13T16:50:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd098c00700 1 --2- 172.21.15.3:0/853016577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd09415b660 0x7fd0940b2a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd088009880 tx=0x7fd088009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e3fc0255df8802 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd093400700 1 -- 172.21.15.3:0/853016577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088011830 con 0x7fd09415b660 2024-09-13T16:50:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd093400700 1 -- 172.21.15.3:0/853016577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd088011990 con 0x7fd09415b660 2024-09-13T16:50:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd093400700 1 -- 172.21.15.3:0/853016577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088011d00 con 0x7fd09415b660 2024-09-13T16:50:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd09b600700 1 -- 172.21.15.3:0/853016577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd09415b660 msgr2=0x7fd0940b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd09b600700 1 --2- 172.21.15.3:0/853016577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd09415b660 0x7fd0940b2a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd088009880 tx=0x7fd088009b90 comp rx=0 tx=0).stop 2024-09-13T16:50:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd09b600700 1 -- 172.21.15.3:0/853016577 shutdown_connections 2024-09-13T16:50:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd09b600700 1 --2- 172.21.15.3:0/853016577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940714c0 0x7fd09406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd09b600700 1 --2- 172.21.15.3:0/853016577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd09415b660 0x7fd0940b2a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd09b600700 1 -- 172.21.15.3:0/853016577 >> 172.21.15.3:0/853016577 conn(0x7fd09406a390 msgr2=0x7fd09406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd09b600700 1 -- 172.21.15.3:0/853016577 shutdown_connections 2024-09-13T16:50:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.289+0000 7fd09b600700 1 -- 172.21.15.3:0/853016577 wait complete. 2024-09-13T16:50:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.293+0000 7fd09b600700 1 Processor -- start 2024-09-13T16:50:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.293+0000 7fd09b600700 1 -- start start 2024-09-13T16:50:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.293+0000 7fd09b600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940714c0 0x7fd094159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.293+0000 7fd09b600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0940b2f40 0x7fd0941f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd09b600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09415b660 0x7fd094161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd09b600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd094151330 con 0x7fd0940b2f40 2024-09-13T16:50:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd09b600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd094151610 con 0x7fd0940714c0 2024-09-13T16:50:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd09b600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd094159360 con 0x7fd09415b660 2024-09-13T16:50:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0940b2f40 0x7fd0941f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0940b2f40 0x7fd0941f3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44474/0 (socket says 172.21.15.3:44474) 2024-09-13T16:50:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 -- 172.21.15.3:0/4178436575 learned_addr learned my addr 172.21.15.3:0/4178436575 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd098c00700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940714c0 0x7fd094159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd099600700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09415b660 0x7fd094161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 -- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09415b660 msgr2=0x7fd094161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09415b660 0x7fd094161780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 -- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940714c0 msgr2=0x7fd094159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940714c0 0x7fd094159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 -- 172.21.15.3:0/4178436575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880094b0 con 0x7fd0940b2f40 2024-09-13T16:50:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd093e00700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0940b2f40 0x7fd0941f3e80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c4f0 tx=0x7fd08000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd090c00700 1 -- 172.21.15.3:0/4178436575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080005d90 con 0x7fd0940b2f40 2024-09-13T16:50:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd090c00700 1 -- 172.21.15.3:0/4178436575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0800070d0 con 0x7fd0940b2f40 2024-09-13T16:50:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd094159640 con 0x7fd0940b2f40 2024-09-13T16:50:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd090c00700 1 -- 172.21.15.3:0/4178436575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080007440 con 0x7fd0940b2f40 2024-09-13T16:50:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd094150f50 con 0x7fd0940b2f40 2024-09-13T16:50:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.297+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094159b60 con 0x7fd0940b2f40 2024-09-13T16:50:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.301+0000 7fd090c00700 1 -- 172.21.15.3:0/4178436575 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0800075a0 con 0x7fd0940b2f40 2024-09-13T16:50:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.305+0000 7fd090c00700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd074066760 0x7fd074068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.305+0000 7fd090c00700 1 -- 172.21.15.3:0/4178436575 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd0800837c0 con 0x7fd0940b2f40 2024-09-13T16:50:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.305+0000 7fd098c00700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd074066760 0x7fd074068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.305+0000 7fd098c00700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd074066760 0x7fd074068c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd088009bd0 tx=0x7fd08801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.309+0000 7fd090c00700 1 -- 172.21.15.3:0/4178436575 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd094159b60 con 0x7fd0940b2f40 2024-09-13T16:50:35.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.545+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd094159b60 con 0x7fd0940b2f40 2024-09-13T16:50:35.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.545+0000 7fd090c00700 1 -- 172.21.15.3:0/4178436575 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4727 (secure 0 0 0) 0x7fd094159b60 con 0x7fd0940b2f40 2024-09-13T16:50:35.551 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:35.551 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:35.079971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T16:50:35.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd074066760 msgr2=0x7fd074068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd074066760 0x7fd074068c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd088009bd0 tx=0x7fd08801c040 comp rx=0 tx=0).stop 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0940b2f40 msgr2=0x7fd0941f3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0940b2f40 0x7fd0941f3e80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c4f0 tx=0x7fd08000c800 comp rx=0 tx=0).stop 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 shutdown_connections 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd074066760 0x7fd074068c20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09415b660 0x7fd094161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0940b2f40 0x7fd0941f3e80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 --2- 172.21.15.3:0/4178436575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940714c0 0x7fd094159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 >> 172.21.15.3:0/4178436575 conn(0x7fd09406a390 msgr2=0x7fd0940cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 shutdown_connections 2024-09-13T16:50:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.549+0000 7fd09b600700 1 -- 172.21.15.3:0/4178436575 wait complete. 2024-09-13T16:50:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2024-09-13T16:50:35.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.757+0000 7f093e000700 1 Processor -- start 2024-09-13T16:50:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.757+0000 7f093e000700 1 -- start start 2024-09-13T16:50:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0938159840 0x7f093815d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093e000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f093806ccf0 0x7f093806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093e000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0938070280 con 0x7f09380cca20 2024-09-13T16:50:35.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093e000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093806daa0 con 0x7f093806ccf0 2024-09-13T16:50:35.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093e000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09380cce40 con 0x7f0938159840 2024-09-13T16:50:35.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093d600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f093806ccf0 0x7f093806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093d600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f093806ccf0 0x7f093806d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44490/0 (socket says 172.21.15.3:44490) 2024-09-13T16:50:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093d600700 1 -- 172.21.15.3:0/2540483294 learned_addr learned my addr 172.21.15.3:0/2540483294 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.761+0000 7f093cc00700 1 --2- 172.21.15.3:0/2540483294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0938159840 0x7f093815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093d600700 1 -- 172.21.15.3:0/2540483294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0938159840 msgr2=0x7f093815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093d600700 1 --2- 172.21.15.3:0/2540483294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0938159840 0x7f093815d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093d600700 1 -- 172.21.15.3:0/2540483294 >> v1:172.21.15.3:6789/0 conn(0x7f09380cca20 legacy=0x7f093806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093d600700 1 -- 172.21.15.3:0/2540483294 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093815dc00 con 0x7f093806ccf0 2024-09-13T16:50:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093d600700 1 --2- 172.21.15.3:0/2540483294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f093806ccf0 0x7f093806d560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0930009e50 tx=0x7f093000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9674af8a33910369 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f0937400700 1 -- 172.21.15.3:0/2540483294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000e070 con 0x7f093806ccf0 2024-09-13T16:50:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f0937400700 1 -- 172.21.15.3:0/2540483294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093000b910 con 0x7f093806ccf0 2024-09-13T16:50:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f0937400700 1 -- 172.21.15.3:0/2540483294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000bcc0 con 0x7f093806ccf0 2024-09-13T16:50:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093e000700 1 -- 172.21.15.3:0/2540483294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f093806ccf0 msgr2=0x7f093806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093e000700 1 --2- 172.21.15.3:0/2540483294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f093806ccf0 0x7f093806d560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0930009e50 tx=0x7f093000b330 comp rx=0 tx=0).stop 2024-09-13T16:50:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093e000700 1 -- 172.21.15.3:0/2540483294 shutdown_connections 2024-09-13T16:50:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093e000700 1 --2- 172.21.15.3:0/2540483294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f093806ccf0 0x7f093806d560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093e000700 1 --2- 172.21.15.3:0/2540483294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0938159840 0x7f093815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093e000700 1 -- 172.21.15.3:0/2540483294 >> 172.21.15.3:0/2540483294 conn(0x7f09380b0cf0 msgr2=0x7f09380b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093e000700 1 -- 172.21.15.3:0/2540483294 shutdown_connections 2024-09-13T16:50:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.765+0000 7f093e000700 1 -- 172.21.15.3:0/2540483294 wait complete. 2024-09-13T16:50:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093e000700 1 Processor -- start 2024-09-13T16:50:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093e000700 1 -- start start 2024-09-13T16:50:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093806ccf0 0x7f093814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380cca20 0x7f093814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0938159840 0x7f09381f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093e000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09381522f0 con 0x7f0938159840 2024-09-13T16:50:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093e000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09381f35b0 con 0x7f09380cca20 2024-09-13T16:50:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093e000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09381f2ff0 con 0x7f093806ccf0 2024-09-13T16:50:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093d600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0938159840 0x7f09381f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093d600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0938159840 0x7f09381f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44496/0 (socket says 172.21.15.3:44496) 2024-09-13T16:50:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093d600700 1 -- 172.21.15.3:0/1298652237 learned_addr learned my addr 172.21.15.3:0/1298652237 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f093cc00700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093806ccf0 0x7f093814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.769+0000 7f0937e00700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380cca20 0x7f093814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093d600700 1 -- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093806ccf0 msgr2=0x7f093814e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093d600700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093806ccf0 0x7f093814e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093d600700 1 -- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380cca20 msgr2=0x7f093814e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093d600700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380cca20 0x7f093814e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093d600700 1 -- 172.21.15.3:0/1298652237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930009b30 con 0x7f0938159840 2024-09-13T16:50:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093d600700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0938159840 0x7f09381f2680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0938072350 tx=0x7f0930004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f0934c00700 1 -- 172.21.15.3:0/1298652237 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930004800 con 0x7f0938159840 2024-09-13T16:50:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f0934c00700 1 -- 172.21.15.3:0/1298652237 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09300099b0 con 0x7f0938159840 2024-09-13T16:50:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09381f32d0 con 0x7f0938159840 2024-09-13T16:50:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f0934c00700 1 -- 172.21.15.3:0/1298652237 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093001c680 con 0x7f0938159840 2024-09-13T16:50:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093814e5f0 con 0x7f0938159840 2024-09-13T16:50:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.773+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09381524a0 con 0x7f0938159840 2024-09-13T16:50:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.777+0000 7f0934c00700 1 -- 172.21.15.3:0/1298652237 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f093001c7e0 con 0x7f0938159840 2024-09-13T16:50:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.777+0000 7f0934c00700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0918066730 0x7f0918068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.777+0000 7f0934c00700 1 -- 172.21.15.3:0/1298652237 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0930085ff0 con 0x7f0938159840 2024-09-13T16:50:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.777+0000 7f093cc00700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0918066730 0x7f0918068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.781+0000 7f093cc00700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0918066730 0x7f0918068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f092c009b50 tx=0x7f092c009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.785+0000 7f0934c00700 1 -- 172.21.15.3:0/1298652237 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09300b7db0 con 0x7f0938159840 2024-09-13T16:50:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.981+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f093806d560 con 0x7f0938159840 2024-09-13T16:50:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f0934c00700 1 -- 172.21.15.3:0/1298652237 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18311 (secure 0 0 0) 0x7f09381524a0 con 0x7f0938159840 2024-09-13T16:50:35.989 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:35.989 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":24,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:35.079347+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0918066730 msgr2=0x7f0918068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0918066730 0x7f0918068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f092c009b50 tx=0x7f092c009240 comp rx=0 tx=0).stop 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0938159840 msgr2=0x7f09381f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0938159840 0x7f09381f2680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0938072350 tx=0x7f0930004af0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 shutdown_connections 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0918066730 0x7f0918068bf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0938159840 0x7f09381f2680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380cca20 0x7f093814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 --2- 172.21.15.3:0/1298652237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093806ccf0 0x7f093814e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.985+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 >> 172.21.15.3:0/1298652237 conn(0x7f09380b0cf0 msgr2=0x7f09380cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.989+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 shutdown_connections 2024-09-13T16:50:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:35.989+0000 7f093e000700 1 -- 172.21.15.3:0/1298652237 wait complete. 2024-09-13T16:50:36.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f678b200700 1 Processor -- start 2024-09-13T16:50:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f678b200700 1 -- start start 2024-09-13T16:50:36.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f678b200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f678c06fb50 0x7f678c0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f678b200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0cf070 con 0x7f678c0b4560 2024-09-13T16:50:36.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f678b200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0b4be0 con 0x7f678c06ff30 2024-09-13T16:50:36.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f678b200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c15bc20 con 0x7f678c06fb50 2024-09-13T16:50:36.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f678a800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f678c0b4560 0x7f678c15a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48800/0 (socket says 172.21.15.3:48800) 2024-09-13T16:50:36.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f678a800700 1 -- 172.21.15.3:0/3484538477 learned_addr learned my addr 172.21.15.3:0/3484538477 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:36.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.185+0000 7f6789e00700 1 --2- 172.21.15.3:0/3484538477 >> v2:172.21.15.3:3300/0 conn(0x7f678c06fb50 0x7f678c0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789400700 1 -- 172.21.15.3:0/3484538477 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177041648 0 0) 0x7f678c0cf070 con 0x7f678c0b4560 2024-09-13T16:50:36.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789400700 1 -- 172.21.15.3:0/3484538477 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67700035d0 con 0x7f678c0b4560 2024-09-13T16:50:36.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789e00700 1 -- 172.21.15.3:0/3484538477 >> v1:172.21.15.157:6789/0 conn(0x7f678c06ff30 legacy=0x7f678c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789400700 1 -- 172.21.15.3:0/3484538477 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115336987 0 0) 0x7f678c0b4be0 con 0x7f678c06ff30 2024-09-13T16:50:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789e00700 1 -- 172.21.15.3:0/3484538477 >> v1:172.21.15.3:6789/0 conn(0x7f678c0b4560 legacy=0x7f678c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789e00700 1 -- 172.21.15.3:0/3484538477 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0cf330 con 0x7f678c06fb50 2024-09-13T16:50:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789e00700 1 --2- 172.21.15.3:0/3484538477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06fb50 0x7f678c0cfe10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f677c009850 tx=0x7f677c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1237aec74288f082 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789400700 1 -- 172.21.15.3:0/3484538477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c011830 con 0x7f678c06fb50 2024-09-13T16:50:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789400700 1 -- 172.21.15.3:0/3484538477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c011990 con 0x7f678c06fb50 2024-09-13T16:50:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f6789400700 1 -- 172.21.15.3:0/3484538477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c011d40 con 0x7f678c06fb50 2024-09-13T16:50:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f678b200700 1 -- 172.21.15.3:0/3484538477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06fb50 msgr2=0x7f678c0cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f678b200700 1 --2- 172.21.15.3:0/3484538477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06fb50 0x7f678c0cfe10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f677c009850 tx=0x7f677c009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f678b200700 1 -- 172.21.15.3:0/3484538477 shutdown_connections 2024-09-13T16:50:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f678b200700 1 --2- 172.21.15.3:0/3484538477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06fb50 0x7f678c0cfe10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f678b200700 1 -- 172.21.15.3:0/3484538477 >> 172.21.15.3:0/3484538477 conn(0x7f678c0b2910 msgr2=0x7f678c0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f678b200700 1 -- 172.21.15.3:0/3484538477 shutdown_connections 2024-09-13T16:50:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.189+0000 7f678b200700 1 -- 172.21.15.3:0/3484538477 wait complete. 2024-09-13T16:50:36.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f678b200700 1 Processor -- start 2024-09-13T16:50:36.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f678b200700 1 -- start start 2024-09-13T16:50:36.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f678b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678c06fb50 0x7f678c1383d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f678b200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06ff30 0x7f678c06ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f678b200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678c0b4560 0x7f678c1484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f678b200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c138910 con 0x7f678c06ff30 2024-09-13T16:50:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f678b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c147150 con 0x7f678c06fb50 2024-09-13T16:50:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f678b200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c147430 con 0x7f678c0b4560 2024-09-13T16:50:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f6783e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06ff30 0x7f678c06ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f6783e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06ff30 0x7f678c06ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44512/0 (socket says 172.21.15.3:44512) 2024-09-13T16:50:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f6783e00700 1 -- 172.21.15.3:0/2749843246 learned_addr learned my addr 172.21.15.3:0/2749843246 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.193+0000 7f6789e00700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678c06fb50 0x7f678c1383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f678a800700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678c0b4560 0x7f678c1484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6783e00700 1 -- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678c0b4560 msgr2=0x7f678c1484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6783e00700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678c0b4560 0x7f678c1484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6783e00700 1 -- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678c06fb50 msgr2=0x7f678c1383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6783e00700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678c06fb50 0x7f678c1383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6783e00700 1 -- 172.21.15.3:0/2749843246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c0094b0 con 0x7f678c06ff30 2024-09-13T16:50:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6783e00700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06ff30 0x7f678c06ed90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67740011e0 tx=0x7f677400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6782000700 1 -- 172.21.15.3:0/2749843246 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740054e0 con 0x7f678c06ff30 2024-09-13T16:50:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6782000700 1 -- 172.21.15.3:0/2749843246 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67740036a0 con 0x7f678c06ff30 2024-09-13T16:50:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6782000700 1 -- 172.21.15.3:0/2749843246 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774003a50 con 0x7f678c06ff30 2024-09-13T16:50:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c147710 con 0x7f678c06ff30 2024-09-13T16:50:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c147cd0 con 0x7f678c06ff30 2024-09-13T16:50:36.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c0cfe10 con 0x7f678c06ff30 2024-09-13T16:50:36.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.197+0000 7f6782000700 1 -- 172.21.15.3:0/2749843246 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6774003bb0 con 0x7f678c06ff30 2024-09-13T16:50:36.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.201+0000 7f6782000700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6768066730 0x7f6768068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.201+0000 7f6782000700 1 -- 172.21.15.3:0/2749843246 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6774083a40 con 0x7f678c06ff30 2024-09-13T16:50:36.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.201+0000 7f6789e00700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6768066730 0x7f6768068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.201+0000 7f6789e00700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6768066730 0x7f6768068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f677c0090b0 tx=0x7f677c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.205+0000 7f6782000700 1 -- 172.21.15.3:0/2749843246 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6774050a20 con 0x7f678c06ff30 2024-09-13T16:50:36.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.389+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f678c149b60 con 0x7f678c06ff30 2024-09-13T16:50:36.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.393+0000 7f6782000700 1 -- 172.21.15.3:0/2749843246 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7f6774017020 con 0x7f678c06ff30 2024-09-13T16:50:36.397 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:36.397 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:36.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6768066730 msgr2=0x7f6768068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6768066730 0x7f6768068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f677c0090b0 tx=0x7f677c008030 comp rx=0 tx=0).stop 2024-09-13T16:50:36.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06ff30 msgr2=0x7f678c06ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06ff30 0x7f678c06ed90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67740011e0 tx=0x7f677400cae0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 shutdown_connections 2024-09-13T16:50:36.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6768066730 0x7f6768068bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678c0b4560 0x7f678c1484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678c06ff30 0x7f678c06ed90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 --2- 172.21.15.3:0/2749843246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678c06fb50 0x7f678c1383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 >> 172.21.15.3:0/2749843246 conn(0x7f678c0b2910 msgr2=0x7f678c0b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:36.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 shutdown_connections 2024-09-13T16:50:36.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.397+0000 7f678b200700 1 -- 172.21.15.3:0/2749843246 wait complete. 2024-09-13T16:50:36.441 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:36.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.605+0000 7fa117e00700 1 Processor -- start 2024-09-13T16:50:36.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.609+0000 7fa117e00700 1 -- start start 2024-09-13T16:50:36.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.609+0000 7fa117e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa11015dec0 0x7fa110160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.609+0000 7fa117e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa1100b5b90 0x7fa11006fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.609+0000 7fa117e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1100b5730 con 0x7fa11015cad0 2024-09-13T16:50:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100b6000 con 0x7fa11015dec0 2024-09-13T16:50:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100cde50 con 0x7fa1100b5b90 2024-09-13T16:50:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa114a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa11015dec0 0x7fa110160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa114a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa11015dec0 0x7fa110160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44528/0 (socket says 172.21.15.3:44528) 2024-09-13T16:50:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa114a00700 1 -- 172.21.15.3:0/3575235947 learned_addr learned my addr 172.21.15.3:0/3575235947 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa115e00700 1 --2- 172.21.15.3:0/3575235947 >> v2:172.21.15.157:3300/0 conn(0x7fa1100b5b90 0x7fa11006fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa114a00700 1 -- 172.21.15.3:0/3575235947 >> v2:172.21.15.157:3300/0 conn(0x7fa1100b5b90 msgr2=0x7fa11006fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa114a00700 1 --2- 172.21.15.3:0/3575235947 >> v2:172.21.15.157:3300/0 conn(0x7fa1100b5b90 0x7fa11006fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa114a00700 1 -- 172.21.15.3:0/3575235947 >> v1:172.21.15.157:6789/0 conn(0x7fa11015cad0 legacy=0x7fa1100cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa114a00700 1 -- 172.21.15.3:0/3575235947 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100b3e70 con 0x7fa11015dec0 2024-09-13T16:50:36.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa114a00700 1 --2- 172.21.15.3:0/3575235947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa110160920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1100d0710 tx=0x7fa0fc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0b2951e55f482c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa10fe00700 1 -- 172.21.15.3:0/3575235947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00b8a0 con 0x7fa11015dec0 2024-09-13T16:50:36.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa10fe00700 1 -- 172.21.15.3:0/3575235947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc00ba00 con 0x7fa11015dec0 2024-09-13T16:50:36.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa10fe00700 1 -- 172.21.15.3:0/3575235947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00bd90 con 0x7fa11015dec0 2024-09-13T16:50:36.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 -- 172.21.15.3:0/3575235947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 msgr2=0x7fa110160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 --2- 172.21.15.3:0/3575235947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa110160920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1100d0710 tx=0x7fa0fc009bf0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 -- 172.21.15.3:0/3575235947 shutdown_connections 2024-09-13T16:50:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 --2- 172.21.15.3:0/3575235947 >> v2:172.21.15.157:3300/0 conn(0x7fa1100b5b90 0x7fa11006fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 --2- 172.21.15.3:0/3575235947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa110160920 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 -- 172.21.15.3:0/3575235947 >> 172.21.15.3:0/3575235947 conn(0x7fa1100b2ad0 msgr2=0x7fa1100b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:36.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 -- 172.21.15.3:0/3575235947 shutdown_connections 2024-09-13T16:50:36.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.613+0000 7fa117e00700 1 -- 172.21.15.3:0/3575235947 wait complete. 2024-09-13T16:50:36.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.617+0000 7fa117e00700 1 Processor -- start 2024-09-13T16:50:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.617+0000 7fa117e00700 1 -- start start 2024-09-13T16:50:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.617+0000 7fa117e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1100b5b90 0x7fa110146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.617+0000 7fa117e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa11015cad0 0x7fa110159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.617+0000 7fa117e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa1100cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.617+0000 7fa117e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa110155830 con 0x7fa11015dec0 2024-09-13T16:50:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.617+0000 7fa117e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa110154c20 con 0x7fa1100b5b90 2024-09-13T16:50:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.617+0000 7fa117e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa110154f00 con 0x7fa11015cad0 2024-09-13T16:50:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa1100cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa114a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa11015cad0 0x7fa110159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa1100cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44530/0 (socket says 172.21.15.3:44530) 2024-09-13T16:50:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 -- 172.21.15.3:0/3593894524 learned_addr learned my addr 172.21.15.3:0/3593894524 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115400700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1100b5b90 0x7fa110146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 -- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa11015cad0 msgr2=0x7fa110159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa11015cad0 0x7fa110159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 -- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1100b5b90 msgr2=0x7fa110146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1100b5b90 0x7fa110146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 -- 172.21.15.3:0/3593894524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc009570 con 0x7fa11015dec0 2024-09-13T16:50:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa115e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa1100cba00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa108007e60 tx=0x7fa10800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa10d600700 1 -- 172.21.15.3:0/3593894524 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10800ed30 con 0x7fa11015dec0 2024-09-13T16:50:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa10d600700 1 -- 172.21.15.3:0/3593894524 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa108014420 con 0x7fa11015dec0 2024-09-13T16:50:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1101551e0 con 0x7fa11015dec0 2024-09-13T16:50:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa10d600700 1 -- 172.21.15.3:0/3593894524 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1080147d0 con 0x7fa11015dec0 2024-09-13T16:50:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa110146bd0 con 0x7fa11015dec0 2024-09-13T16:50:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.621+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa110151a60 con 0x7fa11015dec0 2024-09-13T16:50:36.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.625+0000 7fa10d600700 1 -- 172.21.15.3:0/3593894524 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa108014930 con 0x7fa11015dec0 2024-09-13T16:50:36.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.625+0000 7fa10d600700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0f0066760 0x7fa0f0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.625+0000 7fa10d600700 1 -- 172.21.15.3:0/3593894524 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa108084a40 con 0x7fa11015dec0 2024-09-13T16:50:36.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.629+0000 7fa115400700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0f0066760 0x7fa0f0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.629+0000 7fa115400700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0f0066760 0x7fa0f0068c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa104000f60 tx=0x7fa104009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.633+0000 7fa10d600700 1 -- 172.21.15.3:0/3593894524 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa108051ab0 con 0x7fa11015dec0 2024-09-13T16:50:36.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.869+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa110160920 con 0x7fa11015dec0 2024-09-13T16:50:36.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.873+0000 7fa10d600700 1 -- 172.21.15.3:0/3593894524 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4727 (secure 0 0 0) 0x7fa1080550d0 con 0x7fa11015dec0 2024-09-13T16:50:36.877 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:36.877 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:35.079971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0f0066760 msgr2=0x7fa0f0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0f0066760 0x7fa0f0068c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa104000f60 tx=0x7fa104009040 comp rx=0 tx=0).stop 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 msgr2=0x7fa1100cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa1100cba00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa108007e60 tx=0x7fa10800e3c0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 shutdown_connections 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0f0066760 0x7fa0f0068c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa11015dec0 0x7fa1100cba00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa11015cad0 0x7fa110159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 --2- 172.21.15.3:0/3593894524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1100b5b90 0x7fa110146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 >> 172.21.15.3:0/3593894524 conn(0x7fa1100b2ad0 msgr2=0x7fa1100b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:36.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 shutdown_connections 2024-09-13T16:50:36.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:36.877+0000 7fa117e00700 1 -- 172.21.15.3:0/3593894524 wait complete. 2024-09-13T16:50:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2024-09-13T16:50:36.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:37.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.069+0000 7fde5da00700 1 Processor -- start 2024-09-13T16:50:37.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.069+0000 7fde5da00700 1 -- start start 2024-09-13T16:50:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde5da00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde580ce3b0 0x7fde5815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde5da00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 0x7fde5806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde5da00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde580b2c90 con 0x7fde5806d0e0 2024-09-13T16:50:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde5da00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5806da90 con 0x7fde580ce3b0 2024-09-13T16:50:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde5da00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde580724c0 con 0x7fde580720c0 2024-09-13T16:50:37.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde4fe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fde5806d0e0 0x7fde58160950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48806/0 (socket says 172.21.15.3:48806) 2024-09-13T16:50:37.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde4fe00700 1 -- 172.21.15.3:0/3579890225 learned_addr learned my addr 172.21.15.3:0/3579890225 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:37.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde56a00700 1 --2- 172.21.15.3:0/3579890225 >> v2:172.21.15.3:3300/0 conn(0x7fde580ce3b0 0x7fde5815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde57400700 1 --2- 172.21.15.3:0/3579890225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 0x7fde5806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde56a00700 1 -- 172.21.15.3:0/3579890225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 msgr2=0x7fde5806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde56a00700 1 --2- 172.21.15.3:0/3579890225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 0x7fde5806d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde56000700 1 -- 172.21.15.3:0/3579890225 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941688373 0 0) 0x7fde580b2c90 con 0x7fde5806d0e0 2024-09-13T16:50:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde56a00700 1 -- 172.21.15.3:0/3579890225 >> v1:172.21.15.3:6789/0 conn(0x7fde5806d0e0 legacy=0x7fde58160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.073+0000 7fde56a00700 1 -- 172.21.15.3:0/3579890225 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5815e320 con 0x7fde580ce3b0 2024-09-13T16:50:37.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde56a00700 1 --2- 172.21.15.3:0/3579890225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815dde0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde48009850 tx=0x7fde48009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467c37b8706cd3bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde56000700 1 -- 172.21.15.3:0/3579890225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48011830 con 0x7fde580ce3b0 2024-09-13T16:50:37.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde56000700 1 -- 172.21.15.3:0/3579890225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde48011990 con 0x7fde580ce3b0 2024-09-13T16:50:37.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde56000700 1 -- 172.21.15.3:0/3579890225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48011d20 con 0x7fde580ce3b0 2024-09-13T16:50:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 -- 172.21.15.3:0/3579890225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 msgr2=0x7fde5815dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 --2- 172.21.15.3:0/3579890225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815dde0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde48009850 tx=0x7fde48009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 -- 172.21.15.3:0/3579890225 shutdown_connections 2024-09-13T16:50:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 --2- 172.21.15.3:0/3579890225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 0x7fde5806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 --2- 172.21.15.3:0/3579890225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815dde0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 -- 172.21.15.3:0/3579890225 >> 172.21.15.3:0/3579890225 conn(0x7fde5806a390 msgr2=0x7fde5806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 -- 172.21.15.3:0/3579890225 shutdown_connections 2024-09-13T16:50:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 -- 172.21.15.3:0/3579890225 wait complete. 2024-09-13T16:50:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.077+0000 7fde5da00700 1 Processor -- start 2024-09-13T16:50:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 -- start start 2024-09-13T16:50:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde5806d0e0 0x7fde58159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 0x7fde580cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5815ade0 con 0x7fde580ce3b0 2024-09-13T16:50:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde58142eb0 con 0x7fde5806d0e0 2024-09-13T16:50:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde581422a0 con 0x7fde580720c0 2024-09-13T16:50:37.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44546/0 (socket says 172.21.15.3:44546) 2024-09-13T16:50:37.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 -- 172.21.15.3:0/1705724453 learned_addr learned my addr 172.21.15.3:0/1705724453 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:37.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde56a00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde5806d0e0 0x7fde58159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde4fe00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 0x7fde580cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 -- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 msgr2=0x7fde580cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 0x7fde580cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 -- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde5806d0e0 msgr2=0x7fde58159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde5806d0e0 0x7fde58159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 -- 172.21.15.3:0/1705724453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480094b0 con 0x7fde580ce3b0 2024-09-13T16:50:37.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde57400700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815a8a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde50007d90 tx=0x7fde5000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde4ea00700 1 -- 172.21.15.3:0/1705724453 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5000ed30 con 0x7fde580ce3b0 2024-09-13T16:50:37.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde4ea00700 1 -- 172.21.15.3:0/1705724453 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde50014420 con 0x7fde580ce3b0 2024-09-13T16:50:37.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde4ea00700 1 -- 172.21.15.3:0/1705724453 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde500147d0 con 0x7fde580ce3b0 2024-09-13T16:50:37.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde58142580 con 0x7fde580ce3b0 2024-09-13T16:50:37.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.081+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde58142b40 con 0x7fde580ce3b0 2024-09-13T16:50:37.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.085+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde58143060 con 0x7fde580ce3b0 2024-09-13T16:50:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.085+0000 7fde4ea00700 1 -- 172.21.15.3:0/1705724453 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde50014930 con 0x7fde580ce3b0 2024-09-13T16:50:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.089+0000 7fde4ea00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fde34066730 0x7fde34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.089+0000 7fde4ea00700 1 -- 172.21.15.3:0/1705724453 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fde50084b00 con 0x7fde580ce3b0 2024-09-13T16:50:37.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.089+0000 7fde56a00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fde34066730 0x7fde34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.089+0000 7fde56a00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fde34066730 0x7fde34068bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fde48014040 tx=0x7fde48008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.093+0000 7fde4ea00700 1 -- 172.21.15.3:0/1705724453 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fde58143060 con 0x7fde580ce3b0 2024-09-13T16:50:37.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.289+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde58143020 con 0x7fde580ce3b0 2024-09-13T16:50:37.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.289+0000 7fde4ea00700 1 -- 172.21.15.3:0/1705724453 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7fde50086e60 con 0x7fde580ce3b0 2024-09-13T16:50:37.297 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:37.297 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fde34066730 msgr2=0x7fde34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fde34066730 0x7fde34068bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fde48014040 tx=0x7fde48008030 comp rx=0 tx=0).stop 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 msgr2=0x7fde5815a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815a8a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde50007d90 tx=0x7fde5000e3c0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 shutdown_connections 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fde34066730 0x7fde34068bf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde580ce3b0 0x7fde5815a8a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde580720c0 0x7fde580cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 --2- 172.21.15.3:0/1705724453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde5806d0e0 0x7fde58159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 >> 172.21.15.3:0/1705724453 conn(0x7fde5806a390 msgr2=0x7fde5815fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 shutdown_connections 2024-09-13T16:50:37.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.293+0000 7fde5da00700 1 -- 172.21.15.3:0/1705724453 wait complete. 2024-09-13T16:50:37.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:50:30.569564+0000', 'modified': '2024-09-13T16:50:35.079971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6837/3884968743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 3884968743}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 3884968743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-09-13T16:50:37.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:50:37.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-13T16:50:37.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.901+0000 7fcf1be00700 1 Processor -- start 2024-09-13T16:50:37.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.901+0000 7fcf1be00700 1 -- start start 2024-09-13T16:50:37.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf1c1443b0 0x7fcf1c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1be00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf1c144790 0x7fcf1c06c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1be00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1c0ce8e0 con 0x7fcf1c06cb20 2024-09-13T16:50:37.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1be00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1c06cfb0 con 0x7fcf1c1443b0 2024-09-13T16:50:37.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1be00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1c0ccd30 con 0x7fcf1c144790 2024-09-13T16:50:37.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1aa00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf1c1443b0 0x7fcf1c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1aa00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf1c1443b0 0x7fcf1c0b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44552/0 (socket says 172.21.15.3:44552) 2024-09-13T16:50:37.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1aa00700 1 -- 172.21.15.3:0/3210847625 learned_addr learned my addr 172.21.15.3:0/3210847625 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:37.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1a000700 1 --2- 172.21.15.3:0/3210847625 >> v2:172.21.15.157:3300/0 conn(0x7fcf1c144790 0x7fcf1c06c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1aa00700 1 -- 172.21.15.3:0/3210847625 >> v2:172.21.15.157:3300/0 conn(0x7fcf1c144790 msgr2=0x7fcf1c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/3210847625 >> v2:172.21.15.157:3300/0 conn(0x7fcf1c144790 0x7fcf1c06c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1aa00700 1 -- 172.21.15.3:0/3210847625 >> v1:172.21.15.157:6789/0 conn(0x7fcf1c06cb20 legacy=0x7fcf1c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1aa00700 1 -- 172.21.15.3:0/3210847625 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c0cd010 con 0x7fcf1c1443b0 2024-09-13T16:50:37.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/3210847625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c1443b0 0x7fcf1c0b39f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c06fc60 tx=0x7fcf0c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf4b85b43173f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf19600700 1 -- 172.21.15.3:0/3210847625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c004230 con 0x7fcf1c1443b0 2024-09-13T16:50:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf19600700 1 -- 172.21.15.3:0/3210847625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c004390 con 0x7fcf1c1443b0 2024-09-13T16:50:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf19600700 1 -- 172.21.15.3:0/3210847625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0036c0 con 0x7fcf1c1443b0 2024-09-13T16:50:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1be00700 1 -- 172.21.15.3:0/3210847625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c1443b0 msgr2=0x7fcf1c0b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.905+0000 7fcf1be00700 1 --2- 172.21.15.3:0/3210847625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c1443b0 0x7fcf1c0b39f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c06fc60 tx=0x7fcf0c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.909+0000 7fcf1be00700 1 -- 172.21.15.3:0/3210847625 shutdown_connections 2024-09-13T16:50:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.909+0000 7fcf1be00700 1 --2- 172.21.15.3:0/3210847625 >> v2:172.21.15.157:3300/0 conn(0x7fcf1c144790 0x7fcf1c06c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.909+0000 7fcf1be00700 1 --2- 172.21.15.3:0/3210847625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c1443b0 0x7fcf1c0b39f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.909+0000 7fcf1be00700 1 -- 172.21.15.3:0/3210847625 >> 172.21.15.3:0/3210847625 conn(0x7fcf1c0b0cf0 msgr2=0x7fcf1c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.909+0000 7fcf1be00700 1 -- 172.21.15.3:0/3210847625 shutdown_connections 2024-09-13T16:50:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.909+0000 7fcf1be00700 1 -- 172.21.15.3:0/3210847625 wait complete. 2024-09-13T16:50:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.909+0000 7fcf1be00700 1 Processor -- start 2024-09-13T16:50:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.909+0000 7fcf1be00700 1 -- start start 2024-09-13T16:50:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c06cb20 0x7fcf1c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf1c1443b0 0x7fcf1c13cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c159f40 0x7fcf1c156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1be00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c159b20 con 0x7fcf1c159f40 2024-09-13T16:50:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c13d430 con 0x7fcf1c06cb20 2024-09-13T16:50:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1be00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c158ea0 con 0x7fcf1c1443b0 2024-09-13T16:50:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c06cb20 0x7fcf1c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c06cb20 0x7fcf1c1587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:57446/0 (socket says 172.21.15.3:57446) 2024-09-13T16:50:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 -- 172.21.15.3:0/4118970220 learned_addr learned my addr 172.21.15.3:0/4118970220 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1b400700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c159f40 0x7fcf1c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1a000700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf1c1443b0 0x7fcf1c13cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 -- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf1c1443b0 msgr2=0x7fcf1c13cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf1c1443b0 0x7fcf1c13cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 -- 172.21.15.3:0/4118970220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c159f40 msgr2=0x7fcf1c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c159f40 0x7fcf1c156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 -- 172.21.15.3:0/4118970220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c009570 con 0x7fcf1c06cb20 2024-09-13T16:50:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1b400700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c159f40 0x7fcf1c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c06cb20 0x7fcf1c1587a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c004000 tx=0x7fcf0c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf12a00700 1 -- 172.21.15.3:0/4118970220 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c003cf0 con 0x7fcf1c06cb20 2024-09-13T16:50:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf12a00700 1 -- 172.21.15.3:0/4118970220 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c01f070 con 0x7fcf1c06cb20 2024-09-13T16:50:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf12a00700 1 -- 172.21.15.3:0/4118970220 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c011870 con 0x7fcf1c06cb20 2024-09-13T16:50:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1be00700 1 -- 172.21.15.3:0/4118970220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c159180 con 0x7fcf1c06cb20 2024-09-13T16:50:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.913+0000 7fcf1be00700 1 -- 172.21.15.3:0/4118970220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c159740 con 0x7fcf1c06cb20 2024-09-13T16:50:37.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.917+0000 7fcf12a00700 1 -- 172.21.15.3:0/4118970220 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf0c021030 con 0x7fcf1c06cb20 2024-09-13T16:50:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf12a00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcefc066730 0x7fcefc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf12a00700 1 -- 172.21.15.3:0/4118970220 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcf0c0858c0 con 0x7fcf1c06cb20 2024-09-13T16:50:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf1a000700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcefc066730 0x7fcefc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf24d7b740 1 Processor -- start 2024-09-13T16:50:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf24d7b740 1 -- start start 2024-09-13T16:50:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf1a000700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcefc066730 0x7fcefc068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c15a530 tx=0x7fcf08009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc19910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc41330 0x3cc62fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc634e0 0x3cc638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x3cb86d80 con 0x3cc19530 2024-09-13T16:50:37.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x3cb84510 con 0x3cc634e0 2024-09-13T16:50:37.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.921+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3cb7fe00 con 0x3cc41330 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc19910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1a000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc41330 0x3cc62fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc19910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44568/0 (socket says 172.21.15.3:44568) 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 -- 172.21.15.3:0/464555656 learned_addr learned my addr 172.21.15.3:0/464555656 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1b400700 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc634e0 0x3cc638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 -- 172.21.15.3:0/464555656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc41330 msgr2=0x3cc62fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc41330 0x3cc62fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 -- 172.21.15.3:0/464555656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc634e0 msgr2=0x3cc638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc634e0 0x3cc638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 -- 172.21.15.3:0/464555656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x3c9f2960 con 0x3cc19530 2024-09-13T16:50:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1a000700 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc41330 0x3cc62fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:37.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc19910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c056580 tx=0x7fcf0c055fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e02832b75d98d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf07400700 1 -- 172.21.15.3:0/464555656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c056cf0 con 0x3cc19530 2024-09-13T16:50:37.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf07400700 1 -- 172.21.15.3:0/464555656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c0565f0 con 0x3cc19530 2024-09-13T16:50:37.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf07400700 1 -- 172.21.15.3:0/464555656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c05a420 con 0x3cc19530 2024-09-13T16:50:37.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 -- 172.21.15.3:0/464555656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 msgr2=0x3cc19910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc19910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c056580 tx=0x7fcf0c055fd0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 -- 172.21.15.3:0/464555656 shutdown_connections 2024-09-13T16:50:37.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc634e0 0x3cc638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc41330 0x3cc62fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/464555656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc19910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 -- 172.21.15.3:0/464555656 >> 172.21.15.3:0/464555656 conn(0x3c9a74d0 msgr2=0x3cc40c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:37.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 -- 172.21.15.3:0/464555656 shutdown_connections 2024-09-13T16:50:37.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.925+0000 7fcf24d7b740 1 -- 172.21.15.3:0/464555656 wait complete. 2024-09-13T16:50:37.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 1 -- start start 2024-09-13T16:50:37.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 1 Processor -- start 2024-09-13T16:50:37.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc3df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3cc3ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x3cb3d730 con 0x3cc19530 2024-09-13T16:50:37.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x3cb435b0 con 0x3cc41330 2024-09-13T16:50:37.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3ca2baa0 con 0x3cc634e0 2024-09-13T16:50:37.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf24d7b740 10 client.0 _ll_register_callbacks cb 0x3cb79610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:50:37.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf1aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc3df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf1aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc3df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44570/0 (socket says 172.21.15.3:44570) 2024-09-13T16:50:37.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf1aa00700 1 -- 172.21.15.3:0/3330597924 learned_addr learned my addr 172.21.15.3:0/3330597924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:37.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf1a000700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.929+0000 7fcf1b400700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3cc3ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1aa00700 1 -- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 msgr2=0x3cc3ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3cc3ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1aa00700 1 -- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 msgr2=0x3cc3a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1aa00700 1 -- 172.21.15.3:0/3330597924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf08004180 con 0x3cc19530 2024-09-13T16:50:37.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1b400700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3cc3ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:37.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1a000700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:37.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc3df80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c024000 tx=0x7fcf0c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd6149da06fe684 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcefbe00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcefbe00700 1 -- 172.21.15.3:0/3330597924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c06e040 con 0x3cc19530 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcefbe00700 1 -- 172.21.15.3:0/3330597924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcf0c056cf0 con 0x3cc19530 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 1 -- 172.21.15.3:0/3330597924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x3c9b41e0 con 0x3cc19530 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcefbe00700 1 -- 172.21.15.3:0/3330597924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0c05e9e0 con 0x3cc19530 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 10 client.4419 fetch_fsmap learned FSMap version 6 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 1 -- 172.21.15.3:0/3330597924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x7fcf1c159740 con 0x3cc19530 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcefbe00700 1 -- 172.21.15.3:0/3330597924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x3c9b41e0 con 0x3cc19530 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcefbe00700 1 -- 172.21.15.3:0/3330597924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fcf0c078070 con 0x3cc19530 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 10 client.4419 fetch_fsmap finished waiting for FSMap version 6 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 10 client.4419 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 20 client.4419 populate_metadata read hostname 'smithi003' 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x3cc61ee0 0x3cc622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 4 client.4419 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:50:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf24d7b740 1 -- 172.21.15.3:0/3330597924 --> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x3c9fe460 con 0x3cc61ee0 2024-09-13T16:50:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.933+0000 7fcf1a000700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x3cc61ee0 0x3cc622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcefbe00700 10 client.4419 ms_handle_connect on v2:172.21.15.157:6834/3884968743 2024-09-13T16:50:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf1a000700 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x3cc61ee0 0x3cc622c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f254a3b95c69822 server_cookie=bcd5f1df10a017ff in_seq=0 out_seq=0 2024-09-13T16:50:37.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcefbe00700 1 -- 172.21.15.3:0/3330597924 <== mds.0 v2:172.21.15.157:6834/3884968743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x3c9fe460 con 0x3cc61ee0 2024-09-13T16:50:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcefbe00700 10 client.4419 handle_command_reply: tid=0 2024-09-13T16:50:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 client.4419 shutdown 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 -- 172.21.15.3:0/3330597924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 msgr2=0x3cc3df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc3df80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c024000 tx=0x7fcf0c00b260 comp rx=0 tx=0).stop 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 -- 172.21.15.3:0/3330597924 shutdown_connections 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x3cc61ee0 0x3cc622c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3cc3ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcefbe00700 0 client.4419 ms_handle_reset on v2:172.21.15.157:6834/3884968743 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/3330597924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc19530 0x3cc3df80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 -- 172.21.15.3:0/3330597924 >> 172.21.15.3:0/3330597924 conn(0x3c9a74d0 msgr2=0x3cc40c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 -- 172.21.15.3:0/3330597924 shutdown_connections 2024-09-13T16:50:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 1 -- 172.21.15.3:0/3330597924 wait complete. 2024-09-13T16:50:37.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.937+0000 7fcf24d7b740 20 client.4419 trim_cache size 0 max 16384 2024-09-13T16:50:37.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.961+0000 7fcf24d7b740 1 Processor -- start 2024-09-13T16:50:37.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.961+0000 7fcf24d7b740 1 -- start start 2024-09-13T16:50:37.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc19530 0x3cc3b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc634e0 0x3cc3be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x3cb84510 con 0x3cc634e0 2024-09-13T16:50:37.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x3cbaeec0 con 0x3cc41330 2024-09-13T16:50:37.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3cb3d730 con 0x3cc19530 2024-09-13T16:50:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc634e0 0x3cc3be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc634e0 0x3cc3be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44584/0 (socket says 172.21.15.3:44584) 2024-09-13T16:50:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 -- 172.21.15.3:0/808406551 learned_addr learned my addr 172.21.15.3:0/808406551 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1a000700 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc19530 0x3cc3b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 -- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc19530 msgr2=0x3cc3b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc19530 0x3cc3b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 -- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 msgr2=0x3cc3b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 -- 172.21.15.3:0/808406551 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x3cb46220 con 0x3cc634e0 2024-09-13T16:50:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1a000700 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf1b400700 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc634e0 0x3cc3be70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf14014810 tx=0x7fcf14004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c1925f6bdc8eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf07e00700 1 -- 172.21.15.3:0/808406551 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140049f0 con 0x3cc634e0 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf07e00700 1 -- 172.21.15.3:0/808406551 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf14004b70 con 0x3cc634e0 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf07e00700 1 -- 172.21.15.3:0/808406551 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140167b0 con 0x3cc634e0 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf24d7b740 1 -- 172.21.15.3:0/808406551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc634e0 msgr2=0x3cc3be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.965+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc634e0 0x3cc3be70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf14014810 tx=0x7fcf14004260 comp rx=0 tx=0).stop 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 -- 172.21.15.3:0/808406551 shutdown_connections 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc634e0 0x3cc3be70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc41330 0x3cc3b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/808406551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc19530 0x3cc3b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 -- 172.21.15.3:0/808406551 >> 172.21.15.3:0/808406551 conn(0x3c9a74d0 msgr2=0x3cc5e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 -- 172.21.15.3:0/808406551 shutdown_connections 2024-09-13T16:50:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 -- 172.21.15.3:0/808406551 wait complete. 2024-09-13T16:50:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 -- start start 2024-09-13T16:50:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 Processor -- start 2024-09-13T16:50:37.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc19530 0x3ccc4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc41330 0x3ccc4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3ccc5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x3cb1e470 con 0x3cc41330 2024-09-13T16:50:37.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x3cb435b0 con 0x3cc19530 2024-09-13T16:50:37.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3cc81980 con 0x3cc634e0 2024-09-13T16:50:37.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf24d7b740 10 client.0 _ll_register_callbacks cb 0x7fcf08009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:50:37.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf1a000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc41330 0x3ccc4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf1a000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc41330 0x3ccc4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44598/0 (socket says 172.21.15.3:44598) 2024-09-13T16:50:37.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf1a000700 1 -- 172.21.15.3:0/4126862102 learned_addr learned my addr 172.21.15.3:0/4126862102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:37.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf1b400700 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3ccc5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.969+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc19530 0x3ccc4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf1a000700 1 -- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 msgr2=0x3ccc5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf1a000700 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3ccc5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf1a000700 1 -- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc19530 msgr2=0x3ccc4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf1a000700 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc19530 0x3ccc4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf1a000700 1 -- 172.21.15.3:0/4126862102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf08004310 con 0x3cc41330 2024-09-13T16:50:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf1a000700 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc41330 0x3ccc4f50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf080115a0 tx=0x7fcf080047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac6008a4f85d44f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcefbe00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:50:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcefbe00700 1 -- 172.21.15.3:0/4126862102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0802a070 con 0x3cc41330 2024-09-13T16:50:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcefbe00700 1 -- 172.21.15.3:0/4126862102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcf0802a3e0 con 0x3cc41330 2024-09-13T16:50:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 1 -- 172.21.15.3:0/4126862102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf0c0565f0 con 0x3cc41330 2024-09-13T16:50:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcefbe00700 1 -- 172.21.15.3:0/4126862102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0801c040 con 0x3cc41330 2024-09-13T16:50:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 10 client.4425 fetch_fsmap learned FSMap version 6 2024-09-13T16:50:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 1 -- 172.21.15.3:0/4126862102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x7fcf14014460 con 0x3cc41330 2024-09-13T16:50:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcefbe00700 1 -- 172.21.15.3:0/4126862102 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c0565f0 con 0x3cc41330 2024-09-13T16:50:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcefbe00700 1 -- 172.21.15.3:0/4126862102 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fcf0802a5a0 con 0x3cc41330 2024-09-13T16:50:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 10 client.4425 fetch_fsmap finished waiting for FSMap version 6 2024-09-13T16:50:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 10 client.4425 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-13T16:50:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 20 client.4425 populate_metadata read hostname 'smithi003' 2024-09-13T16:50:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x3ccc5d30 0x3cc7a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 4 client.4425 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:50:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf24d7b740 1 -- 172.21.15.3:0/4126862102 --> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] -- command(tid 0: {"prefix": "status"}) v1 -- 0x3ca231e0 con 0x3ccc5d30 2024-09-13T16:50:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.973+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x3ccc5d30 0x3cc7a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1aa00700 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x3ccc5d30 0x3cc7a0c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2dfe90ac903af6 server_cookie=182ed221f3107e49 in_seq=0 out_seq=0 2024-09-13T16:50:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcefbe00700 10 client.4425 ms_handle_connect on v2:172.21.15.157:6834/3884968743 2024-09-13T16:50:37.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcefbe00700 1 -- 172.21.15.3:0/4126862102 <== mds.0 v2:172.21.15.157:6834/3884968743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x3ca231e0 con 0x3ccc5d30 2024-09-13T16:50:37.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcefbe00700 10 client.4425 handle_command_reply: tid=0 2024-09-13T16:50:37.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 client.4425 shutdown 2024-09-13T16:50:37.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 -- 172.21.15.3:0/4126862102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc41330 msgr2=0x3ccc4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc41330 0x3ccc4f50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf080115a0 tx=0x7fcf080047e0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 -- 172.21.15.3:0/4126862102 shutdown_connections 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x3ccc5d30 0x3cc7a0c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3cc634e0 0x3ccc5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3cc41330 0x3ccc4f50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcefbe00700 0 client.4425 ms_handle_reset on v2:172.21.15.157:6834/3884968743 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 --2- 172.21.15.3:0/4126862102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3cc19530 0x3ccc4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 -- 172.21.15.3:0/4126862102 >> 172.21.15.3:0/4126862102 conn(0x3c9a74d0 msgr2=0x3cc13700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 -- 172.21.15.3:0/4126862102 shutdown_connections 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 1 -- 172.21.15.3:0/4126862102 wait complete. 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf24d7b740 20 client.4425 trim_cache size 0 max 16384 2024-09-13T16:50:37.983 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "61456116-a42d-4065-9bf5-a1daf0be594e", 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4284, 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 7.4041513029999999, 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 6, 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 25, 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 23, 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 12.045816421 2024-09-13T16:50:37.984 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:50:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 -- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcefc066730 msgr2=0x7fcefc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcefc066730 0x7fcefc068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c15a530 tx=0x7fcf08009240 comp rx=0 tx=0).stop 2024-09-13T16:50:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 -- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c06cb20 msgr2=0x7fcf1c1587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c06cb20 0x7fcf1c1587a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c004000 tx=0x7fcf0c004770 comp rx=0 tx=0).stop 2024-09-13T16:50:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 -- 172.21.15.3:0/4118970220 shutdown_connections 2024-09-13T16:50:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcefc066730 0x7fcefc068bf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c159f40 0x7fcf1c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf1c1443b0 0x7fcf1c13cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 --2- 172.21.15.3:0/4118970220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c06cb20 0x7fcf1c1587a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 -- 172.21.15.3:0/4118970220 >> 172.21.15.3:0/4118970220 conn(0x7fcf1c0b0cf0 msgr2=0x7fcf1c0aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 -- 172.21.15.3:0/4118970220 shutdown_connections 2024-09-13T16:50:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:37.977+0000 7fcf1be00700 1 -- 172.21.15.3:0/4118970220 wait complete. 2024-09-13T16:50:38.007 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:50:38.007 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:50:38.007 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:50:38.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f3757e00700 1 Processor -- start 2024-09-13T16:50:38.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f3757e00700 1 -- start start 2024-09-13T16:50:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f3757e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f375806ccf0 0x7f37581472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f3757e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3758165800 con 0x7f3758146ee0 2024-09-13T16:50:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f3757e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3758164ed0 con 0x7f375806fea0 2024-09-13T16:50:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f3757e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3758070a80 con 0x7f375806ccf0 2024-09-13T16:50:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f3756a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3758146ee0 0x7f375806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48820/0 (socket says 172.21.15.3:48820) 2024-09-13T16:50:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f374fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f375806ccf0 0x7f37581472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:38.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.181+0000 7f3756a00700 1 -- 172.21.15.3:0/3794427172 learned_addr learned my addr 172.21.15.3:0/3794427172 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:38.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3756000700 1 -- 172.21.15.3:0/3794427172 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204723999 0 0) 0x7f3758165800 con 0x7f3758146ee0 2024-09-13T16:50:38.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3756000700 1 -- 172.21.15.3:0/3794427172 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37380035d0 con 0x7f3758146ee0 2024-09-13T16:50:38.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f374fe00700 1 -- 172.21.15.3:0/3794427172 >> v1:172.21.15.157:6789/0 conn(0x7f375806fea0 legacy=0x7f37581616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:38.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f374fe00700 1 -- 172.21.15.3:0/3794427172 >> v1:172.21.15.3:6789/0 conn(0x7f3758146ee0 legacy=0x7f375806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:38.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f374fe00700 1 -- 172.21.15.3:0/3794427172 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37581651b0 con 0x7f375806ccf0 2024-09-13T16:50:38.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f374fe00700 1 --2- 172.21.15.3:0/3794427172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f375806ccf0 0x7f37581472c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3740009850 tx=0x7f3740009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766a7cd1b8a94c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:38.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3756000700 1 -- 172.21.15.3:0/3794427172 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740011830 con 0x7f375806ccf0 2024-09-13T16:50:38.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3756000700 1 -- 172.21.15.3:0/3794427172 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3740011990 con 0x7f375806ccf0 2024-09-13T16:50:38.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3756000700 1 -- 172.21.15.3:0/3794427172 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740011d00 con 0x7f375806ccf0 2024-09-13T16:50:38.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3757e00700 1 -- 172.21.15.3:0/3794427172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f375806ccf0 msgr2=0x7f37581472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3757e00700 1 --2- 172.21.15.3:0/3794427172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f375806ccf0 0x7f37581472c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3740009850 tx=0x7f3740009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3757e00700 1 -- 172.21.15.3:0/3794427172 shutdown_connections 2024-09-13T16:50:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3757e00700 1 --2- 172.21.15.3:0/3794427172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f375806ccf0 0x7f37581472c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3757e00700 1 -- 172.21.15.3:0/3794427172 >> 172.21.15.3:0/3794427172 conn(0x7f37580b0cf0 msgr2=0x7f37580b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3757e00700 1 -- 172.21.15.3:0/3794427172 shutdown_connections 2024-09-13T16:50:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.185+0000 7f3757e00700 1 -- 172.21.15.3:0/3794427172 wait complete. 2024-09-13T16:50:38.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.189+0000 7f3757e00700 1 Processor -- start 2024-09-13T16:50:38.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.189+0000 7f3757e00700 1 -- start start 2024-09-13T16:50:38.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f375806ccf0 0x7f375815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:38.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806fea0 0x7f37581fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:38.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3758146ee0 0x7f37581fb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375815bab0 con 0x7f3758146ee0 2024-09-13T16:50:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37580ae220 con 0x7f375806fea0 2024-09-13T16:50:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375815ffd0 con 0x7f375806ccf0 2024-09-13T16:50:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3758146ee0 0x7f37581fb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3758146ee0 0x7f37581fb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44608/0 (socket says 172.21.15.3:44608) 2024-09-13T16:50:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 -- 172.21.15.3:0/2909601316 learned_addr learned my addr 172.21.15.3:0/2909601316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f374fe00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806fea0 0x7f37581fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3756a00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f375806ccf0 0x7f375815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 -- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f375806ccf0 msgr2=0x7f375815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:38.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f375806ccf0 0x7f375815c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:38.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 -- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806fea0 msgr2=0x7f37581fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:38.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806fea0 0x7f37581fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:38.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 -- 172.21.15.3:0/2909601316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400094b0 con 0x7f3758146ee0 2024-09-13T16:50:38.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f374fe00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806fea0 0x7f37581fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757400700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3758146ee0 0x7f37581fb170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f375000cc50 tx=0x7f375000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f374ea00700 1 -- 172.21.15.3:0/2909601316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750003c20 con 0x7f3758146ee0 2024-09-13T16:50:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f374ea00700 1 -- 172.21.15.3:0/2909601316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3750003d80 con 0x7f3758146ee0 2024-09-13T16:50:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f374ea00700 1 -- 172.21.15.3:0/2909601316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375000aab0 con 0x7f3758146ee0 2024-09-13T16:50:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375815c110 con 0x7f3758146ee0 2024-09-13T16:50:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.193+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37581601e0 con 0x7f3758146ee0 2024-09-13T16:50:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.197+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37581472c0 con 0x7f3758146ee0 2024-09-13T16:50:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.197+0000 7f374ea00700 1 -- 172.21.15.3:0/2909601316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f375000ac10 con 0x7f3758146ee0 2024-09-13T16:50:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.201+0000 7f374ea00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3734066760 0x7f3734068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.201+0000 7f374ea00700 1 -- 172.21.15.3:0/2909601316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3750084520 con 0x7f3758146ee0 2024-09-13T16:50:38.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.201+0000 7f3756a00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3734066760 0x7f3734068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:38.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.201+0000 7f3756a00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3734066760 0x7f3734068c20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3748007fd0 tx=0x7f3748007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:38.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.205+0000 7f374ea00700 1 -- 172.21.15.3:0/2909601316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3750051480 con 0x7f3758146ee0 2024-09-13T16:50:38.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.437+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f37580ae370 con 0x7f3758146ee0 2024-09-13T16:50:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.437+0000 7f374ea00700 1 -- 172.21.15.3:0/2909601316 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3750054aa0 con 0x7f3758146ee0 2024-09-13T16:50:38.442 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T16:50:38.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3734066760 msgr2=0x7f3734068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:38.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3734066760 0x7f3734068c20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3748007fd0 tx=0x7f3748007de0 comp rx=0 tx=0).stop 2024-09-13T16:50:38.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3758146ee0 msgr2=0x7f37581fb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:38.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3758146ee0 0x7f37581fb170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f375000cc50 tx=0x7f375000cf60 comp rx=0 tx=0).stop 2024-09-13T16:50:38.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 shutdown_connections 2024-09-13T16:50:38.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3734066760 0x7f3734068c20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:38.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3758146ee0 0x7f37581fb170 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:38.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806fea0 0x7f37581fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:38.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 --2- 172.21.15.3:0/2909601316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f375806ccf0 0x7f375815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:38.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 >> 172.21.15.3:0/2909601316 conn(0x7f37580b0cf0 msgr2=0x7f37580afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:38.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 shutdown_connections 2024-09-13T16:50:38.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:38.441+0000 7f3757e00700 1 -- 172.21.15.3:0/2909601316 wait complete. 2024-09-13T16:50:38.466 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T16:50:38.467 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:50:38.467 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-13T16:50:38.467 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-13T16:50:38.467 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:50:38.467 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T16:50:38.467 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:50:38.467 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:50:38.467 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T16:50:38.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:38.467 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-13T16:50:38.486 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:50:38.486 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:50:38.486 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:50:38.486 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T16:50:38.487 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42427sec preferred_lft 42427sec 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:4d:e8:d2:67 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:50:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:50:38.489 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T16:50:38.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:38.489 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:50:38.490 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:50:38.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-09-13T16:50:38.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-09-13T16:50:38.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-09-13T16:50:38.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T16:50:38.490 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:50:38.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:38.646 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:50:38.679 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-13T16:50:38.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:38.682 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-13T16:50:38.758 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-13T16:50:38.758 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:50:38.759 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T16:50:38.759 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:50:38.759 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T16:50:38.759 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:50:38.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:38.759 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:50:38.759 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:50:38.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T16:50:38.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T16:50:38.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T16:50:38.760 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:50:38.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:38.896 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:50:38.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:38.912 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-13T16:50:38.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:38.976 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:50:38.976 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:50:38.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T16:50:38.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-13T16:50:38.976 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:50:39.094 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-13T16:50:39.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:39.094 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:50:39.094 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:50:39.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-13T16:50:39.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T16:50:39.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-13T16:50:39.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-13T16:50:39.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-13T16:50:39.095 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:50:39.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:39.261 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:50:39.261 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:50:39.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-13T16:50:39.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T16:50:39.262 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:50:39.336 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-13T16:50:39.336 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:50:39.336 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:50:39.347 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T16:50:39.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:50:39.348 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:50:39.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-13T16:50:39.493 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:50:39.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:50:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:50:39.570 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:50:39.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:39.571 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-13T16:50:39.637 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T16:50:39.637 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-13T16:50:39.685 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-13T16:50:39.722 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:50:39.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:50:39.794 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:50:39.797 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:50:39.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:39.797 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-13T16:50:39.824 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:2024-09-13T16:50:39.821+0000 7fa0fd8c9640 -1 init, newargv = 0x5590a2fc4a70 newargc=15 2024-09-13T16:50:39.824 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[32222]: starting ceph client 2024-09-13T16:50:39.841 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[32222]: starting fuse 2024-09-13T16:50:40.941 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-13T16:50:40.984 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:50:40.985 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:50:41.067 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:50:41.070 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:50:41.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:41.071 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-13T16:50:41.134 INFO:teuthology.orchestra.run.smithi003.stdout:50 2024-09-13T16:50:41.135 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2024-09-13T16:50:41.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:50:41.135 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-13T16:50:41.135 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:50:41.136 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.admin")) 2024-09-13T16:50:41.137 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:50:41.304 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.admin.32222.asok 2024-09-13T16:50:41.485 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.32222.asok 2024-09-13T16:50:41.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:41.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.32222.asok status 2024-09-13T16:50:41.689 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:50:41.689 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-13T16:50:41.689 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "50bbe7f395997dcd871813949c49addbbf630361", 2024-09-13T16:50:41.689 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)", 2024-09-13T16:50:41.689 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "admin", 2024-09-13T16:50:41.689 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-13T16:50:41.689 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-13T16:50:41.689 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "32222", 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4440, 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 4440 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:50:41.690 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3840842215 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3840842215 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.4440 192.168.0.1:0/3840842215", 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3840842215", 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 6, 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 25, 2024-09-13T16:50:41.691 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-13T16:50:41.692 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-13T16:50:41.692 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-13T16:50:41.692 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:50:41.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:41.704 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-13T16:50:41.705 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:50:41.725 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-13T16:50:41.726 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:50:41.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:41.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:50:41.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:41.799 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-13T16:50:41.903 INFO:teuthology.orchestra.run.smithi003.stdout:# file: . 2024-09-13T16:50:41.904 INFO:teuthology.orchestra.run.smithi003.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-13T16:50:41.904 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:41.905 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:41.906 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:50:41.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:42.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.141+0000 7f5f47e00700 1 Processor -- start 2024-09-13T16:50:42.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.141+0000 7f5f47e00700 1 -- start start 2024-09-13T16:50:42.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.141+0000 7f5f47e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f48146ee0 0x7f5f481472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.141+0000 7f5f47e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5f4806c5e0 0x7f5f4806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f47e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f48165800 con 0x7f5f4806fea0 2024-09-13T16:50:42.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f47e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f48070a80 con 0x7f5f4806c5e0 2024-09-13T16:50:42.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f47e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f481626e0 con 0x7f5f48146ee0 2024-09-13T16:50:42.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f3fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5f4806c5e0 0x7f5f4806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f46a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f48146ee0 0x7f5f481472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f47400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5f4806fea0 0x7f5f4806a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:50120/0 (socket says 172.21.15.3:50120) 2024-09-13T16:50:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f47400700 1 -- 172.21.15.3:0/2770520299 learned_addr learned my addr 172.21.15.3:0/2770520299 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f3fe00700 1 -- 172.21.15.3:0/2770520299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f48146ee0 msgr2=0x7f5f481472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f46000700 1 -- 172.21.15.3:0/2770520299 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864757861 0 0) 0x7f5f48165800 con 0x7f5f4806fea0 2024-09-13T16:50:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f3fe00700 1 --2- 172.21.15.3:0/2770520299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f48146ee0 0x7f5f481472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f3fe00700 1 -- 172.21.15.3:0/2770520299 >> v1:172.21.15.157:6789/0 conn(0x7f5f4806fea0 legacy=0x7f5f4806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f3fe00700 1 -- 172.21.15.3:0/2770520299 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4806cf60 con 0x7f5f4806c5e0 2024-09-13T16:50:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f46a00700 1 --2- 172.21.15.3:0/2770520299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f48146ee0 0x7f5f481472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f3fe00700 1 --2- 172.21.15.3:0/2770520299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f4806ca20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009850 tx=0x7f5f30009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3ca699c8cc15d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f46000700 1 -- 172.21.15.3:0/2770520299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30011830 con 0x7f5f4806c5e0 2024-09-13T16:50:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f46000700 1 -- 172.21.15.3:0/2770520299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30011990 con 0x7f5f4806c5e0 2024-09-13T16:50:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.145+0000 7f5f46000700 1 -- 172.21.15.3:0/2770520299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30011d40 con 0x7f5f4806c5e0 2024-09-13T16:50:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.149+0000 7f5f47e00700 1 -- 172.21.15.3:0/2770520299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 msgr2=0x7f5f4806ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.149+0000 7f5f47e00700 1 --2- 172.21.15.3:0/2770520299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f4806ca20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009850 tx=0x7f5f30009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.149+0000 7f5f47e00700 1 -- 172.21.15.3:0/2770520299 shutdown_connections 2024-09-13T16:50:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.149+0000 7f5f47e00700 1 --2- 172.21.15.3:0/2770520299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f4806ca20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.149+0000 7f5f47e00700 1 --2- 172.21.15.3:0/2770520299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f48146ee0 0x7f5f481472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.149+0000 7f5f47e00700 1 -- 172.21.15.3:0/2770520299 >> 172.21.15.3:0/2770520299 conn(0x7f5f480b0cf0 msgr2=0x7f5f480b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.149+0000 7f5f47e00700 1 -- 172.21.15.3:0/2770520299 shutdown_connections 2024-09-13T16:50:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.149+0000 7f5f47e00700 1 -- 172.21.15.3:0/2770520299 wait complete. 2024-09-13T16:50:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47e00700 1 Processor -- start 2024-09-13T16:50:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47e00700 1 -- start start 2024-09-13T16:50:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f48163470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4806fea0 0x7f5f48141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f48146ee0 0x7f5f4815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815dc10 con 0x7f5f4806c5e0 2024-09-13T16:50:42.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815cf90 con 0x7f5f48146ee0 2024-09-13T16:50:42.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815d270 con 0x7f5f4806fea0 2024-09-13T16:50:42.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f46a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f48163470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f46a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f48163470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44626/0 (socket says 172.21.15.3:44626) 2024-09-13T16:50:42.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f46a00700 1 -- 172.21.15.3:0/350461600 learned_addr learned my addr 172.21.15.3:0/350461600 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:42.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f3fe00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4806fea0 0x7f5f48141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47400700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f48146ee0 0x7f5f4815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f46a00700 1 -- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4806fea0 msgr2=0x7f5f48141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f46a00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4806fea0 0x7f5f48141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f46a00700 1 -- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f48146ee0 msgr2=0x7f5f4815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f46a00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f48146ee0 0x7f5f4815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f46a00700 1 -- 172.21.15.3:0/350461600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380094b0 con 0x7f5f4806c5e0 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f47400700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f48146ee0 0x7f5f4815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.153+0000 7f5f3fe00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4806fea0 0x7f5f48141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.157+0000 7f5f46a00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f48163470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800ebe0 tx=0x7f5f38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.157+0000 7f5f3ea00700 1 -- 172.21.15.3:0/350461600 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800c8b0 con 0x7f5f4806c5e0 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.157+0000 7f5f3ea00700 1 -- 172.21.15.3:0/350461600 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3800ca10 con 0x7f5f4806c5e0 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.157+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f300094b0 con 0x7f5f4806c5e0 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.157+0000 7f5f3ea00700 1 -- 172.21.15.3:0/350461600 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800cda0 con 0x7f5f4806c5e0 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.157+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4815d830 con 0x7f5f4806c5e0 2024-09-13T16:50:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.157+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4806ca20 con 0x7f5f4806c5e0 2024-09-13T16:50:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.161+0000 7f5f3ea00700 1 -- 172.21.15.3:0/350461600 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f38013470 con 0x7f5f4806c5e0 2024-09-13T16:50:42.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.161+0000 7f5f3ea00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5f24066730 0x7f5f24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.161+0000 7f5f3ea00700 1 -- 172.21.15.3:0/350461600 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5f38084580 con 0x7f5f4806c5e0 2024-09-13T16:50:42.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.161+0000 7f5f3fe00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5f24066730 0x7f5f24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.165+0000 7f5f3fe00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5f24066730 0x7f5f24068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009850 tx=0x7f5f30008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:42.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.169+0000 7f5f3ea00700 1 -- 172.21.15.3:0/350461600 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5f380515f0 con 0x7f5f4806c5e0 2024-09-13T16:50:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.353+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f480b40b0 con 0x7f5f4806c5e0 2024-09-13T16:50:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.353+0000 7f5f3ea00700 1 -- 172.21.15.3:0/350461600 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7f5f38054c10 con 0x7f5f4806c5e0 2024-09-13T16:50:42.360 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:42.360 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.357+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5f24066730 msgr2=0x7f5f24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.357+0000 7f5f47e00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5f24066730 0x7f5f24068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009850 tx=0x7f5f30008030 comp rx=0 tx=0).stop 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 msgr2=0x7f5f48163470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f48163470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800ebe0 tx=0x7f5f38009e30 comp rx=0 tx=0).stop 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 shutdown_connections 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5f24066730 0x7f5f24068bf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f48146ee0 0x7f5f4815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4806fea0 0x7f5f48141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 --2- 172.21.15.3:0/350461600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f4806c5e0 0x7f5f48163470 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 >> 172.21.15.3:0/350461600 conn(0x7f5f480b0cf0 msgr2=0x7f5f480cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 shutdown_connections 2024-09-13T16:50:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.361+0000 7f5f47e00700 1 -- 172.21.15.3:0/350461600 wait complete. 2024-09-13T16:50:42.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:50:42.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:50:42.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.565+0000 7f7d47200700 1 Processor -- start 2024-09-13T16:50:42.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.565+0000 7f7d47200700 1 -- start start 2024-09-13T16:50:42.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d47200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d4015b1c0 0x7f7d400ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d47200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d400cc830 0x7f7d400ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d47200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d400cd1c0 con 0x7f7d40158740 2024-09-13T16:50:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d47200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d400b3960 con 0x7f7d4015b1c0 2024-09-13T16:50:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d47200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d400b1f30 con 0x7f7d400cc830 2024-09-13T16:50:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d45200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d400cc830 0x7f7d400ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3f400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d4015b1c0 0x7f7d400ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3fe00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7d40158740 0x7f7d400b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:50124/0 (socket says 172.21.15.3:50124) 2024-09-13T16:50:42.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3fe00700 1 -- 172.21.15.3:0/1390904083 learned_addr learned my addr 172.21.15.3:0/1390904083 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:42.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3f400700 1 -- 172.21.15.3:0/1390904083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d400cc830 msgr2=0x7f7d400ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:42.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3f400700 1 --2- 172.21.15.3:0/1390904083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d400cc830 0x7f7d400ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3f400700 1 -- 172.21.15.3:0/1390904083 >> v1:172.21.15.157:6789/0 conn(0x7f7d40158740 legacy=0x7f7d400b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3f400700 1 -- 172.21.15.3:0/1390904083 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4015b710 con 0x7f7d4015b1c0 2024-09-13T16:50:42.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3ea00700 1 -- 172.21.15.3:0/1390904083 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417650835 0 0) 0x7f7d30001110 con 0x7f7d40158740 2024-09-13T16:50:42.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.569+0000 7f7d3f400700 1 --2- 172.21.15.3:0/1390904083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d4015b1c0 0x7f7d400ce6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009850 tx=0x7f7d2c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=750d1d728bdddae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:42.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d3ea00700 1 -- 172.21.15.3:0/1390904083 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c011830 con 0x7f7d4015b1c0 2024-09-13T16:50:42.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d3ea00700 1 -- 172.21.15.3:0/1390904083 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c011990 con 0x7f7d4015b1c0 2024-09-13T16:50:42.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d3ea00700 1 -- 172.21.15.3:0/1390904083 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c011d40 con 0x7f7d4015b1c0 2024-09-13T16:50:42.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 -- 172.21.15.3:0/1390904083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d4015b1c0 msgr2=0x7f7d400ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 --2- 172.21.15.3:0/1390904083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d4015b1c0 0x7f7d400ce6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009850 tx=0x7f7d2c009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:42.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 -- 172.21.15.3:0/1390904083 shutdown_connections 2024-09-13T16:50:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 --2- 172.21.15.3:0/1390904083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d400cc830 0x7f7d400ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 --2- 172.21.15.3:0/1390904083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d4015b1c0 0x7f7d400ce6e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 -- 172.21.15.3:0/1390904083 >> 172.21.15.3:0/1390904083 conn(0x7f7d4006b280 msgr2=0x7f7d4006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 -- 172.21.15.3:0/1390904083 shutdown_connections 2024-09-13T16:50:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 -- 172.21.15.3:0/1390904083 wait complete. 2024-09-13T16:50:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 Processor -- start 2024-09-13T16:50:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.573+0000 7f7d47200700 1 -- start start 2024-09-13T16:50:42.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d47200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d400cc830 0x7f7d40070120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d47200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d40158740 0x7f7d4014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d47200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d4015b1c0 0x7f7d4015cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d47200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40070700 con 0x7f7d4015b1c0 2024-09-13T16:50:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d47200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d400709e0 con 0x7f7d400cc830 2024-09-13T16:50:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d47200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40070cc0 con 0x7f7d40158740 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d4015b1c0 0x7f7d4015cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d4015b1c0 0x7f7d4015cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44632/0 (socket says 172.21.15.3:44632) 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 -- 172.21.15.3:0/2575107188 learned_addr learned my addr 172.21.15.3:0/2575107188 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d3f400700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d40158740 0x7f7d4014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d3fe00700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d400cc830 0x7f7d40070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 -- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d40158740 msgr2=0x7f7d4014a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d40158740 0x7f7d4014a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 -- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d400cc830 msgr2=0x7f7d40070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d400cc830 0x7f7d40070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 -- 172.21.15.3:0/2575107188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c0094b0 con 0x7f7d4015b1c0 2024-09-13T16:50:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d45200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d4015b1c0 0x7f7d4015cc50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800efc0 tx=0x7f7d3800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d37e00700 1 -- 172.21.15.3:0/2575107188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800cad0 con 0x7f7d4015b1c0 2024-09-13T16:50:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d37e00700 1 -- 172.21.15.3:0/2575107188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800f040 con 0x7f7d4015b1c0 2024-09-13T16:50:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d40070fa0 con 0x7f7d4015b1c0 2024-09-13T16:50:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d37e00700 1 -- 172.21.15.3:0/2575107188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800ce70 con 0x7f7d4015b1c0 2024-09-13T16:50:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.577+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d401554c0 con 0x7f7d4015b1c0 2024-09-13T16:50:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.581+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d400714c0 con 0x7f7d4015b1c0 2024-09-13T16:50:42.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.581+0000 7f7d37e00700 1 -- 172.21.15.3:0/2575107188 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d38014480 con 0x7f7d4015b1c0 2024-09-13T16:50:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.581+0000 7f7d37e00700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d20066760 0x7f7d20068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.585+0000 7f7d37e00700 1 -- 172.21.15.3:0/2575107188 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7d38084ba0 con 0x7f7d4015b1c0 2024-09-13T16:50:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.585+0000 7f7d3fe00700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d20066760 0x7f7d20068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.585+0000 7f7d3fe00700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d20066760 0x7f7d20068c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d30000f60 tx=0x7f7d30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:42.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.593+0000 7f7d37e00700 1 -- 172.21.15.3:0/2575107188 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7d400714c0 con 0x7f7d4015b1c0 2024-09-13T16:50:42.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.777+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d400714c0 con 0x7f7d4015b1c0 2024-09-13T16:50:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.777+0000 7f7d37e00700 1 -- 172.21.15.3:0/2575107188 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d38018020 con 0x7f7d4015b1c0 2024-09-13T16:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:50:42.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d20066760 msgr2=0x7f7d20068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d20066760 0x7f7d20068c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d30000f60 tx=0x7f7d30009040 comp rx=0 tx=0).stop 2024-09-13T16:50:42.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d4015b1c0 msgr2=0x7f7d4015cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:42.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d4015b1c0 0x7f7d4015cc50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800efc0 tx=0x7f7d3800c0c0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 shutdown_connections 2024-09-13T16:50:42.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d20066760 0x7f7d20068c20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d4015b1c0 0x7f7d4015cc50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d40158740 0x7f7d4014a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 --2- 172.21.15.3:0/2575107188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d400cc830 0x7f7d40070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:42.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 >> 172.21.15.3:0/2575107188 conn(0x7f7d4006b280 msgr2=0x7f7d4015dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:42.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 shutdown_connections 2024-09-13T16:50:42.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.781+0000 7f7d47200700 1 -- 172.21.15.3:0/2575107188 wait complete. 2024-09-13T16:50:42.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:50:42.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.989+0000 7fa459400700 1 Processor -- start 2024-09-13T16:50:42.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.989+0000 7fa459400700 1 -- start start 2024-09-13T16:50:42.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa459400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa45406cfb0 0x7fa45406b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:42.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa459400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4540cedf0 con 0x7fa454160ef0 2024-09-13T16:50:42.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa459400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4540cf0d0 con 0x7fa45406c360 2024-09-13T16:50:42.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa459400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4540cfa70 con 0x7fa45406cfb0 2024-09-13T16:50:42.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa453e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa454160ef0 0x7fa4540cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48822/0 (socket says 172.21.15.3:48822) 2024-09-13T16:50:42.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa453e00700 1 -- 172.21.15.3:0/4142125552 learned_addr learned my addr 172.21.15.3:0/4142125552 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:42.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa453400700 1 --2- 172.21.15.3:0/4142125552 >> v2:172.21.15.157:3300/0 conn(0x7fa45406cfb0 0x7fa45406b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:42.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324586031 0 0) 0x7fa4540cedf0 con 0x7fa454160ef0 2024-09-13T16:50:42.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.993+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4380035d0 con 0x7fa454160ef0 2024-09-13T16:50:43.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3670618372 0 0) 0x7fa4380035d0 con 0x7fa454160ef0 2024-09-13T16:50:43.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 >> v2:172.21.15.157:3300/0 conn(0x7fa45406cfb0 msgr2=0x7fa45406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:43.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 --2- 172.21.15.3:0/4142125552 >> v2:172.21.15.157:3300/0 conn(0x7fa45406cfb0 0x7fa45406b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 >> v1:172.21.15.157:6789/0 conn(0x7fa45406c360 legacy=0x7fa4541612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540cf3b0 con 0x7fa454160ef0 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fa444001670 con 0x7fa454160ef0 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453400700 1 --2- 172.21.15.3:0/4142125552 >> v2:172.21.15.157:3300/0 conn(0x7fa45406cfb0 0x7fa45406b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 >> v1:172.21.15.3:6789/0 conn(0x7fa454160ef0 legacy=0x7fa4540cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa438003a20 0x7fa438003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438004870 0x7fa438004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438005230 0x7fa43800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540cfa70 con 0x7fa438004870 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43800c180 con 0x7fa438005230 2024-09-13T16:50:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43800c340 con 0x7fa438003a20 2024-09-13T16:50:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453e00700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa438003a20 0x7fa438003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa458a00700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438005230 0x7fa43800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453400700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438004870 0x7fa438004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453e00700 1 -- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438005230 msgr2=0x7fa43800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453e00700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa438005230 0x7fa43800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453e00700 1 -- 172.21.15.3:0/4142125552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438004870 msgr2=0x7fa438004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453e00700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438004870 0x7fa438004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453e00700 1 -- 172.21.15.3:0/4142125552 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380035d0 con 0x7fa438003a20 2024-09-13T16:50:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa458a00700 1 -- 172.21.15.3:0/4142125552 reap_dead start 2024-09-13T16:50:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa453e00700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa438003a20 0x7fa438003e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c320 tx=0x7fa4380058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e19d02550f97e521 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444007c30 con 0x7fa438003a20 2024-09-13T16:50:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44400ce80 con 0x7fa438003a20 2024-09-13T16:50:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:42.997+0000 7fa452a00700 1 -- 172.21.15.3:0/4142125552 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444005bc0 con 0x7fa438003a20 2024-09-13T16:50:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa438003a20 msgr2=0x7fa438003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa438003a20 0x7fa438003e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c320 tx=0x7fa4380058a0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- 172.21.15.3:0/4142125552 shutdown_connections 2024-09-13T16:50:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 --2- 172.21.15.3:0/4142125552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa438003a20 0x7fa438003e40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- 172.21.15.3:0/4142125552 >> 172.21.15.3:0/4142125552 conn(0x7fa4540721a0 msgr2=0x7fa45406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- 172.21.15.3:0/4142125552 shutdown_connections 2024-09-13T16:50:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- 172.21.15.3:0/4142125552 wait complete. 2024-09-13T16:50:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 Processor -- start 2024-09-13T16:50:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- start start 2024-09-13T16:50:43.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438003a20 0x7fa4540cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4540c69a0 0x7fa4540c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4540c8f00 0x7fa4540c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45415b620 con 0x7fa438003a20 2024-09-13T16:50:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540c98c0 con 0x7fa4540c8f00 2024-09-13T16:50:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.001+0000 7fa459400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540c8070 con 0x7fa4540c69a0 2024-09-13T16:50:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438003a20 0x7fa4540cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438003a20 0x7fa4540cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44646/0 (socket says 172.21.15.3:44646) 2024-09-13T16:50:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 -- 172.21.15.3:0/577283492 learned_addr learned my addr 172.21.15.3:0/577283492 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4540c69a0 0x7fa4540c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa458a00700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4540c8f00 0x7fa4540c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 -- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4540c69a0 msgr2=0x7fa4540c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4540c69a0 0x7fa4540c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 -- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4540c8f00 msgr2=0x7fa4540c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4540c8f00 0x7fa4540c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 -- 172.21.15.3:0/577283492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540c7460 con 0x7fa438003a20 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa458a00700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4540c8f00 0x7fa4540c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa453e00700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438003a20 0x7fa4540cc610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa444000f50 tx=0x7fa444001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa44be00700 1 -- 172.21.15.3:0/577283492 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444001630 con 0x7fa438003a20 2024-09-13T16:50:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa44be00700 1 -- 172.21.15.3:0/577283492 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4440070d0 con 0x7fa438003a20 2024-09-13T16:50:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4540c7740 con 0x7fa438003a20 2024-09-13T16:50:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa44be00700 1 -- 172.21.15.3:0/577283492 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4440074a0 con 0x7fa438003a20 2024-09-13T16:50:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4540c7d00 con 0x7fa438003a20 2024-09-13T16:50:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.005+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45406c790 con 0x7fa438003a20 2024-09-13T16:50:43.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.009+0000 7fa44be00700 1 -- 172.21.15.3:0/577283492 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa444007600 con 0x7fa438003a20 2024-09-13T16:50:43.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.009+0000 7fa44be00700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa434066730 0x7fa434068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.009+0000 7fa44be00700 1 -- 172.21.15.3:0/577283492 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa444085650 con 0x7fa438003a20 2024-09-13T16:50:43.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.009+0000 7fa453400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa434066730 0x7fa434068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.013+0000 7fa453400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa434066730 0x7fa434068bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4541f9f60 tx=0x7fa440008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.017+0000 7fa44be00700 1 -- 172.21.15.3:0/577283492 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa444052640 con 0x7fa438003a20 2024-09-13T16:50:43.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.197+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4540cd260 con 0x7fa438003a20 2024-09-13T16:50:43.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.197+0000 7fa44be00700 1 -- 172.21.15.3:0/577283492 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa444055c60 con 0x7fa438003a20 2024-09-13T16:50:43.203 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:50:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa434066730 msgr2=0x7fa434068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa434066730 0x7fa434068bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4541f9f60 tx=0x7fa440008040 comp rx=0 tx=0).stop 2024-09-13T16:50:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438003a20 msgr2=0x7fa4540cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438003a20 0x7fa4540cc610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa444000f50 tx=0x7fa444001a00 comp rx=0 tx=0).stop 2024-09-13T16:50:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 shutdown_connections 2024-09-13T16:50:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa434066730 0x7fa434068bf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4540c8f00 0x7fa4540c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4540c69a0 0x7fa4540c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 --2- 172.21.15.3:0/577283492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa438003a20 0x7fa4540cc610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 >> 172.21.15.3:0/577283492 conn(0x7fa4540721a0 msgr2=0x7fa45406e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 shutdown_connections 2024-09-13T16:50:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.201+0000 7fa459400700 1 -- 172.21.15.3:0/577283492 wait complete. 2024-09-13T16:50:43.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:50:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.397+0000 7f075f000700 1 Processor -- start 2024-09-13T16:50:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.397+0000 7f075f000700 1 -- start start 2024-09-13T16:50:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.397+0000 7f075f000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f075815ab10 0x7f075806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.397+0000 7f075f000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075806f970 0x7f075806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.397+0000 7f075f000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07580cfde0 con 0x7f07580b4810 2024-09-13T16:50:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.397+0000 7f075f000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0758070340 con 0x7f075815ab10 2024-09-13T16:50:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.397+0000 7f075f000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07580b4f80 con 0x7f075806f970 2024-09-13T16:50:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0757e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f075815ab10 0x7f075806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0757400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f07580b4810 0x7f07580d0270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48832/0 (socket says 172.21.15.3:48832) 2024-09-13T16:50:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0757400700 1 -- 172.21.15.3:0/3745537371 learned_addr learned my addr 172.21.15.3:0/3745537371 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075d000700 1 --2- 172.21.15.3:0/3745537371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075806f970 0x7f075806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0757e00700 1 -- 172.21.15.3:0/3745537371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075806f970 msgr2=0x7f075806fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0757e00700 1 --2- 172.21.15.3:0/3745537371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075806f970 0x7f075806fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0757e00700 1 -- 172.21.15.3:0/3745537371 >> v1:172.21.15.3:6789/0 conn(0x7f07580b4810 legacy=0x7f07580d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0756a00700 1 -- 172.21.15.3:0/3745537371 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379540155 0 0) 0x7f07580cfde0 con 0x7f07580b4810 2024-09-13T16:50:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0757e00700 1 -- 172.21.15.3:0/3745537371 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07580cf790 con 0x7f075815ab10 2024-09-13T16:50:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075d000700 1 --2- 172.21.15.3:0/3745537371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075806f970 0x7f075806fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0757e00700 1 --2- 172.21.15.3:0/3745537371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f075815ab10 0x7f075806f0e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0748009850 tx=0x7f0748009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4140c2a5019383 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0756a00700 1 -- 172.21.15.3:0/3745537371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748011830 con 0x7f075815ab10 2024-09-13T16:50:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0756a00700 1 -- 172.21.15.3:0/3745537371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0748011990 con 0x7f075815ab10 2024-09-13T16:50:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f0756a00700 1 -- 172.21.15.3:0/3745537371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748011d20 con 0x7f075815ab10 2024-09-13T16:50:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075f000700 1 -- 172.21.15.3:0/3745537371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f075815ab10 msgr2=0x7f075806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075f000700 1 --2- 172.21.15.3:0/3745537371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f075815ab10 0x7f075806f0e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0748009850 tx=0x7f0748009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075f000700 1 -- 172.21.15.3:0/3745537371 shutdown_connections 2024-09-13T16:50:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075f000700 1 --2- 172.21.15.3:0/3745537371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075806f970 0x7f075806fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075f000700 1 --2- 172.21.15.3:0/3745537371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f075815ab10 0x7f075806f0e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075f000700 1 -- 172.21.15.3:0/3745537371 >> 172.21.15.3:0/3745537371 conn(0x7f07580b2b00 msgr2=0x7f07580b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075f000700 1 -- 172.21.15.3:0/3745537371 shutdown_connections 2024-09-13T16:50:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.401+0000 7f075f000700 1 -- 172.21.15.3:0/3745537371 wait complete. 2024-09-13T16:50:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.405+0000 7f075f000700 1 Processor -- start 2024-09-13T16:50:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.405+0000 7f075f000700 1 -- start start 2024-09-13T16:50:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.405+0000 7f075f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075806f970 0x7f07581f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.405+0000 7f075f000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07580b4810 0x7f0758153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.405+0000 7f075f000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075815ab10 0x7f07581f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.405+0000 7f075f000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07580ad1a0 con 0x7f07580b4810 2024-09-13T16:50:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.405+0000 7f075f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0758153cb0 con 0x7f075806f970 2024-09-13T16:50:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.405+0000 7f075f000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07581f31d0 con 0x7f075815ab10 2024-09-13T16:50:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07580b4810 0x7f0758153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07580b4810 0x7f0758153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44664/0 (socket says 172.21.15.3:44664) 2024-09-13T16:50:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 -- 172.21.15.3:0/1495366222 learned_addr learned my addr 172.21.15.3:0/1495366222 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757e00700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075806f970 0x7f07581f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f075d000700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075815ab10 0x7f07581f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 -- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075815ab10 msgr2=0x7f07581f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075815ab10 0x7f07581f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 -- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075806f970 msgr2=0x7f07581f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075806f970 0x7f07581f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 -- 172.21.15.3:0/1495366222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480094b0 con 0x7f07580b4810 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f075d000700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075815ab10 0x7f07581f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757e00700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075806f970 0x7f07581f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f0757400700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07580b4810 0x7f0758153770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f074400c5b0 tx=0x7f074400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f074fe00700 1 -- 172.21.15.3:0/1495366222 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744004ca0 con 0x7f07580b4810 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f074fe00700 1 -- 172.21.15.3:0/1495366222 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744004e20 con 0x7f07580b4810 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07580ac9c0 con 0x7f07580b4810 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f074fe00700 1 -- 172.21.15.3:0/1495366222 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744007310 con 0x7f07580b4810 2024-09-13T16:50:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07580ad3b0 con 0x7f07580b4810 2024-09-13T16:50:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.409+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07580b4180 con 0x7f07580b4810 2024-09-13T16:50:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.413+0000 7f074fe00700 1 -- 172.21.15.3:0/1495366222 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0744007470 con 0x7f07580b4810 2024-09-13T16:50:43.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.413+0000 7f074fe00700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0738066760 0x7f0738068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.413+0000 7f074fe00700 1 -- 172.21.15.3:0/1495366222 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0744084250 con 0x7f07580b4810 2024-09-13T16:50:43.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.413+0000 7f0757e00700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0738066760 0x7f0738068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.417+0000 7f0757e00700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0738066760 0x7f0738068c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0748014040 tx=0x7f0748008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.421+0000 7f074fe00700 1 -- 172.21.15.3:0/1495366222 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0744051230 con 0x7f07580b4810 2024-09-13T16:50:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.605+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f075806fe00 con 0x7f07580b4810 2024-09-13T16:50:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f074fe00700 1 -- 172.21.15.3:0/1495366222 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0744007720 con 0x7f07580b4810 2024-09-13T16:50:43.612 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:50:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0738066760 msgr2=0x7f0738068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0738066760 0x7f0738068c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0748014040 tx=0x7f0748008030 comp rx=0 tx=0).stop 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07580b4810 msgr2=0x7f0758153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07580b4810 0x7f0758153770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f074400c5b0 tx=0x7f074400c970 comp rx=0 tx=0).stop 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 shutdown_connections 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0738066760 0x7f0738068c20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f075815ab10 0x7f07581f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07580b4810 0x7f0758153770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 --2- 172.21.15.3:0/1495366222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075806f970 0x7f07581f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 >> 172.21.15.3:0/1495366222 conn(0x7f07580b2b00 msgr2=0x7f07580b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 shutdown_connections 2024-09-13T16:50:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.609+0000 7f075f000700 1 -- 172.21.15.3:0/1495366222 wait complete. 2024-09-13T16:50:43.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:50:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.809+0000 7f74f7200700 1 Processor -- start 2024-09-13T16:50:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.809+0000 7f74f7200700 1 -- start start 2024-09-13T16:50:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f7200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74f0037f70 0x7f74f003b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f7200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f0005e30 con 0x7f74f00e6fe0 2024-09-13T16:50:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f7200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f0006110 con 0x7f74f0053090 2024-09-13T16:50:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f7200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f00383c0 con 0x7f74f0037f70 2024-09-13T16:50:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f5400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f74f00e6fe0 0x7f74f0057360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48704/0 (socket says 172.21.15.3:48704) 2024-09-13T16:50:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f5400700 1 -- 172.21.15.3:0/1754960005 learned_addr learned my addr 172.21.15.3:0/1754960005 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f6800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74f0037f70 0x7f74f003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f6800700 1 -- 172.21.15.3:0/1754960005 >> v1:172.21.15.157:6789/0 conn(0x7f74f0053090 legacy=0x7f74f0053470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f6800700 1 -- 172.21.15.3:0/1754960005 >> v1:172.21.15.3:6789/0 conn(0x7f74f00e6fe0 legacy=0x7f74f0057360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f6800700 1 -- 172.21.15.3:0/1754960005 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f00063f0 con 0x7f74f0037f70 2024-09-13T16:50:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f6800700 1 --2- 172.21.15.3:0/1754960005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f003b920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f74f804ff70 tx=0x7f74f804f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a0f80f677b64c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f4a00700 1 -- 172.21.15.3:0/1754960005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f80674a0 con 0x7f74f0037f70 2024-09-13T16:50:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f4a00700 1 -- 172.21.15.3:0/1754960005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f8067640 con 0x7f74f0037f70 2024-09-13T16:50:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f4a00700 1 -- 172.21.15.3:0/1754960005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f80684b0 con 0x7f74f0037f70 2024-09-13T16:50:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f7200700 1 -- 172.21.15.3:0/1754960005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 msgr2=0x7f74f003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.813+0000 7f74f7200700 1 --2- 172.21.15.3:0/1754960005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f003b920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f74f804ff70 tx=0x7f74f804f640 comp rx=0 tx=0).stop 2024-09-13T16:50:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.817+0000 7f74f7200700 1 -- 172.21.15.3:0/1754960005 shutdown_connections 2024-09-13T16:50:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.817+0000 7f74f7200700 1 --2- 172.21.15.3:0/1754960005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f003b920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.817+0000 7f74f7200700 1 -- 172.21.15.3:0/1754960005 >> 172.21.15.3:0/1754960005 conn(0x7f74f003a5c0 msgr2=0x7f74f003a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.817+0000 7f74f7200700 1 -- 172.21.15.3:0/1754960005 shutdown_connections 2024-09-13T16:50:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.817+0000 7f74f7200700 1 -- 172.21.15.3:0/1754960005 wait complete. 2024-09-13T16:50:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.817+0000 7f74f7200700 1 Processor -- start 2024-09-13T16:50:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f7200700 1 -- start start 2024-09-13T16:50:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f7200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f004da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f7200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0053090 0x7f74f004df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00e6fe0 0x7f74f004fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f7200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00e74a0 con 0x7f74f0037f70 2024-09-13T16:50:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f004e4a0 con 0x7f74f00e6fe0 2024-09-13T16:50:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f7200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f0050ed0 con 0x7f74f0053090 2024-09-13T16:50:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f004da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f004da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52596/0 (socket says 172.21.15.3:52596) 2024-09-13T16:50:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f5e00700 1 -- 172.21.15.3:0/3663099404 learned_addr learned my addr 172.21.15.3:0/3663099404 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f6800700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00e6fe0 0x7f74f004fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f5400700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0053090 0x7f74f004df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f5e00700 1 -- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0053090 msgr2=0x7f74f004df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f5e00700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0053090 0x7f74f004df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f5e00700 1 -- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00e6fe0 msgr2=0x7f74f004fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.821+0000 7f74f5e00700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00e6fe0 0x7f74f004fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.825+0000 7f74f5e00700 1 -- 172.21.15.3:0/3663099404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f8073700 con 0x7f74f0037f70 2024-09-13T16:50:43.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.825+0000 7f74f5e00700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f004da20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f74e800c510 tx=0x7f74e800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.825+0000 7f74ee000700 1 -- 172.21.15.3:0/3663099404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8011790 con 0x7f74f0037f70 2024-09-13T16:50:43.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.825+0000 7f74ee000700 1 -- 172.21.15.3:0/3663099404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e8011910 con 0x7f74f0037f70 2024-09-13T16:50:43.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.825+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f00511b0 con 0x7f74f0037f70 2024-09-13T16:50:43.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.825+0000 7f74ee000700 1 -- 172.21.15.3:0/3663099404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8011cc0 con 0x7f74f0037f70 2024-09-13T16:50:43.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.825+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f0051770 con 0x7f74f0037f70 2024-09-13T16:50:43.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.825+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f003b920 con 0x7f74f0037f70 2024-09-13T16:50:43.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.829+0000 7f74ee000700 1 -- 172.21.15.3:0/3663099404 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74e801c460 con 0x7f74f0037f70 2024-09-13T16:50:43.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.829+0000 7f74ee000700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d8066730 0x7f74d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.829+0000 7f74ee000700 1 -- 172.21.15.3:0/3663099404 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74e8084240 con 0x7f74f0037f70 2024-09-13T16:50:43.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.829+0000 7f74f5400700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d8066730 0x7f74d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.833+0000 7f74f5400700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d8066730 0x7f74d8068bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74f00db1e0 tx=0x7f74e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:43.837+0000 7f74ee000700 1 -- 172.21.15.3:0/3663099404 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74e8051220 con 0x7f74f0037f70 2024-09-13T16:50:44.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.017+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f74f0038a30 con 0x7f74f0037f70 2024-09-13T16:50:44.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74ee000700 1 -- 172.21.15.3:0/3663099404 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f74e8054840 con 0x7f74f0037f70 2024-09-13T16:50:44.024 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:50:44.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d8066730 msgr2=0x7f74d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d8066730 0x7f74d8068bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74f00db1e0 tx=0x7f74e4009040 comp rx=0 tx=0).stop 2024-09-13T16:50:44.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 msgr2=0x7f74f004da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f004da20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f74e800c510 tx=0x7f74e800c820 comp rx=0 tx=0).stop 2024-09-13T16:50:44.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 shutdown_connections 2024-09-13T16:50:44.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d8066730 0x7f74d8068bf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00e6fe0 0x7f74f004fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0053090 0x7f74f004df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 --2- 172.21.15.3:0/3663099404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f0037f70 0x7f74f004da20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 >> 172.21.15.3:0/3663099404 conn(0x7f74f003a5c0 msgr2=0x7f74f0005100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:44.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 shutdown_connections 2024-09-13T16:50:44.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.021+0000 7f74f7200700 1 -- 172.21.15.3:0/3663099404 wait complete. 2024-09-13T16:50:44.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:50:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.225+0000 7f636fe00700 1 Processor -- start 2024-09-13T16:50:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.225+0000 7f636fe00700 1 -- start start 2024-09-13T16:50:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.225+0000 7f636fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f637006ccf0 0x7f637015d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.225+0000 7f636fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f637015ac30 0x7f63700cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.225+0000 7f636fe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6370070280 con 0x7f6370159840 2024-09-13T16:50:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.225+0000 7f636fe00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637015dc00 con 0x7f637006ccf0 2024-09-13T16:50:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.225+0000 7f636fe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637015dee0 con 0x7f637015ac30 2024-09-13T16:50:44.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.225+0000 7f636e000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f637006ccf0 0x7f637015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636e000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f637006ccf0 0x7f637015d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52606/0 (socket says 172.21.15.3:52606) 2024-09-13T16:50:44.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636ea00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6370159840 0x7f637006c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:59714/0 (socket says 172.21.15.3:59714) 2024-09-13T16:50:44.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636e000700 1 -- 172.21.15.3:0/3932446823 learned_addr learned my addr 172.21.15.3:0/3932446823 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:44.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636f400700 1 --2- 172.21.15.3:0/3932446823 >> v2:172.21.15.157:3300/0 conn(0x7f637015ac30 0x7f63700cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636e000700 1 -- 172.21.15.3:0/3932446823 >> v2:172.21.15.157:3300/0 conn(0x7f637015ac30 msgr2=0x7f63700cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636e000700 1 --2- 172.21.15.3:0/3932446823 >> v2:172.21.15.157:3300/0 conn(0x7f637015ac30 0x7f63700cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636e000700 1 -- 172.21.15.3:0/3932446823 >> v1:172.21.15.157:6789/0 conn(0x7f6370159840 legacy=0x7f637006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636e000700 1 -- 172.21.15.3:0/3932446823 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637015e1c0 con 0x7f637006ccf0 2024-09-13T16:50:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636e000700 1 --2- 172.21.15.3:0/3932446823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637006ccf0 0x7f637015d620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63700b2190 tx=0x7f635c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd77cd6e52f8f31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636d600700 1 -- 172.21.15.3:0/3932446823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c00b8a0 con 0x7f637006ccf0 2024-09-13T16:50:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636d600700 1 -- 172.21.15.3:0/3932446823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c00ba00 con 0x7f637006ccf0 2024-09-13T16:50:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636d600700 1 -- 172.21.15.3:0/3932446823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c00bd90 con 0x7f637006ccf0 2024-09-13T16:50:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636fe00700 1 -- 172.21.15.3:0/3932446823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637006ccf0 msgr2=0x7f637015d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636fe00700 1 --2- 172.21.15.3:0/3932446823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637006ccf0 0x7f637015d620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63700b2190 tx=0x7f635c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636fe00700 1 -- 172.21.15.3:0/3932446823 shutdown_connections 2024-09-13T16:50:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636fe00700 1 --2- 172.21.15.3:0/3932446823 >> v2:172.21.15.157:3300/0 conn(0x7f637015ac30 0x7f63700cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636fe00700 1 --2- 172.21.15.3:0/3932446823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637006ccf0 0x7f637015d620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636fe00700 1 -- 172.21.15.3:0/3932446823 >> 172.21.15.3:0/3932446823 conn(0x7f63700b0cf0 msgr2=0x7f63700b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636fe00700 1 -- 172.21.15.3:0/3932446823 shutdown_connections 2024-09-13T16:50:44.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.229+0000 7f636fe00700 1 -- 172.21.15.3:0/3932446823 wait complete. 2024-09-13T16:50:44.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.233+0000 7f636fe00700 1 Processor -- start 2024-09-13T16:50:44.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.233+0000 7f636fe00700 1 -- start start 2024-09-13T16:50:44.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6370159840 0x7f6370152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637015ac30 0x7f6370150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6370146510 0x7f637006a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636fe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63701539d0 con 0x7f637015ac30 2024-09-13T16:50:44.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c9d20 con 0x7f6370159840 2024-09-13T16:50:44.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636fe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63701469a0 con 0x7f6370146510 2024-09-13T16:50:44.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637015ac30 0x7f6370150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637015ac30 0x7f6370150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52612/0 (socket says 172.21.15.3:52612) 2024-09-13T16:50:44.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 -- 172.21.15.3:0/3094381856 learned_addr learned my addr 172.21.15.3:0/3094381856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:44.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636ea00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6370159840 0x7f6370152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636f400700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6370146510 0x7f637006a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 -- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6370146510 msgr2=0x7f637006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6370146510 0x7f637006a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 -- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6370159840 msgr2=0x7f6370152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6370159840 0x7f6370152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 -- 172.21.15.3:0/3094381856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c009570 con 0x7f637015ac30 2024-09-13T16:50:44.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636e000700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637015ac30 0x7f6370150910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f635c013000 tx=0x7f635c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:44.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f6366a00700 1 -- 172.21.15.3:0/3094381856 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c0036c0 con 0x7f637015ac30 2024-09-13T16:50:44.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f6366a00700 1 -- 172.21.15.3:0/3094381856 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c01f070 con 0x7f637015ac30 2024-09-13T16:50:44.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f6366a00700 1 -- 172.21.15.3:0/3094381856 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c003a20 con 0x7f637015ac30 2024-09-13T16:50:44.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6370153410 con 0x7f637015ac30 2024-09-13T16:50:44.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.237+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63700c99b0 con 0x7f637015ac30 2024-09-13T16:50:44.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.241+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63700c9b40 con 0x7f637015ac30 2024-09-13T16:50:44.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.241+0000 7f6366a00700 1 -- 172.21.15.3:0/3094381856 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f635c021030 con 0x7f637015ac30 2024-09-13T16:50:44.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.241+0000 7f6366a00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6350066760 0x7f6350068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.241+0000 7f6366a00700 1 -- 172.21.15.3:0/3094381856 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f635c085850 con 0x7f637015ac30 2024-09-13T16:50:44.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.245+0000 7f636ea00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6350066760 0x7f6350068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.245+0000 7f636ea00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6350066760 0x7f6350068c20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6360000f60 tx=0x7f6360007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:44.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.249+0000 7f6366a00700 1 -- 172.21.15.3:0/3094381856 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f635c052960 con 0x7f637015ac30 2024-09-13T16:50:44.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.445+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f63700c9e90 con 0x7f637015ac30 2024-09-13T16:50:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.445+0000 7f6366a00700 1 -- 172.21.15.3:0/3094381856 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f635c055f80 con 0x7f637015ac30 2024-09-13T16:50:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6350066760 msgr2=0x7f6350068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6350066760 0x7f6350068c20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6360000f60 tx=0x7f6360007f30 comp rx=0 tx=0).stop 2024-09-13T16:50:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637015ac30 msgr2=0x7f6370150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637015ac30 0x7f6370150910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f635c013000 tx=0x7f635c004770 comp rx=0 tx=0).stop 2024-09-13T16:50:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 shutdown_connections 2024-09-13T16:50:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6350066760 0x7f6350068c20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6370146510 0x7f637006a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637015ac30 0x7f6370150910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 --2- 172.21.15.3:0/3094381856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6370159840 0x7f6370152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 >> 172.21.15.3:0/3094381856 conn(0x7f63700b0cf0 msgr2=0x7f63700cb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 shutdown_connections 2024-09-13T16:50:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.453+0000 7f636fe00700 1 -- 172.21.15.3:0/3094381856 wait complete. 2024-09-13T16:50:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-13T16:50:44.479 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:50:44.479 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:50:44.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:44.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.709+0000 7ff687200700 1 Processor -- start 2024-09-13T16:50:44.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.709+0000 7ff687200700 1 -- start start 2024-09-13T16:50:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.709+0000 7ff687200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff68815afa0 0x7ff68806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.709+0000 7ff687200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880b5990 con 0x7ff6880cc300 2024-09-13T16:50:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.709+0000 7ff687200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68815b6f0 con 0x7ff6880cbf20 2024-09-13T16:50:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.709+0000 7ff687200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68815bf90 con 0x7ff68815afa0 2024-09-13T16:50:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff685400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff6880cc300 0x7ff6880b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48708/0 (socket says 172.21.15.3:48708) 2024-09-13T16:50:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff685400700 1 -- 172.21.15.3:0/4058710164 learned_addr learned my addr 172.21.15.3:0/4058710164 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff686800700 1 --2- 172.21.15.3:0/4058710164 >> v2:172.21.15.3:3300/0 conn(0x7ff68815afa0 0x7ff68806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff684a00700 1 -- 172.21.15.3:0/4058710164 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676473358 0 0) 0x7ff6880b5990 con 0x7ff6880cc300 2024-09-13T16:50:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff684a00700 1 -- 172.21.15.3:0/4058710164 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c0035d0 con 0x7ff6880cc300 2024-09-13T16:50:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff684a00700 1 -- 172.21.15.3:0/4058710164 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745536640 0 0) 0x7ff68815b6f0 con 0x7ff6880cbf20 2024-09-13T16:50:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff686800700 1 -- 172.21.15.3:0/4058710164 >> v1:172.21.15.157:6789/0 conn(0x7ff6880cbf20 legacy=0x7ff68806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff686800700 1 -- 172.21.15.3:0/4058710164 >> v1:172.21.15.3:6789/0 conn(0x7ff6880cc300 legacy=0x7ff6880b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff686800700 1 -- 172.21.15.3:0/4058710164 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68815b9d0 con 0x7ff68815afa0 2024-09-13T16:50:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff686800700 1 --2- 172.21.15.3:0/4058710164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff68815afa0 0x7ff68806cc70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff680009850 tx=0x7ff680009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c611707a2d9c4cec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff684a00700 1 -- 172.21.15.3:0/4058710164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680011830 con 0x7ff68815afa0 2024-09-13T16:50:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff684a00700 1 -- 172.21.15.3:0/4058710164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680011990 con 0x7ff68815afa0 2024-09-13T16:50:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff684a00700 1 -- 172.21.15.3:0/4058710164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680011d40 con 0x7ff68815afa0 2024-09-13T16:50:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff687200700 1 -- 172.21.15.3:0/4058710164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff68815afa0 msgr2=0x7ff68806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff687200700 1 --2- 172.21.15.3:0/4058710164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff68815afa0 0x7ff68806cc70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff680009850 tx=0x7ff680009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff687200700 1 -- 172.21.15.3:0/4058710164 shutdown_connections 2024-09-13T16:50:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff687200700 1 --2- 172.21.15.3:0/4058710164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff68815afa0 0x7ff68806cc70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.713+0000 7ff687200700 1 -- 172.21.15.3:0/4058710164 >> 172.21.15.3:0/4058710164 conn(0x7ff688071da0 msgr2=0x7ff6880b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.717+0000 7ff687200700 1 -- 172.21.15.3:0/4058710164 shutdown_connections 2024-09-13T16:50:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.717+0000 7ff687200700 1 -- 172.21.15.3:0/4058710164 wait complete. 2024-09-13T16:50:44.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.717+0000 7ff687200700 1 Processor -- start 2024-09-13T16:50:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.717+0000 7ff687200700 1 -- start start 2024-09-13T16:50:44.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff687200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6880cbf20 0x7ff68814d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff687200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6880cc300 0x7ff6881f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff687200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff68815afa0 0x7ff68814eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff687200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880b47d0 con 0x7ff6880cbf20 2024-09-13T16:50:44.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff687200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881f3830 con 0x7ff68815afa0 2024-09-13T16:50:44.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff687200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68814dbb0 con 0x7ff6880cc300 2024-09-13T16:50:44.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6880cbf20 0x7ff68814d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6880cbf20 0x7ff68814d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52636/0 (socket says 172.21.15.3:52636) 2024-09-13T16:50:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 -- 172.21.15.3:0/2310243574 learned_addr learned my addr 172.21.15.3:0/2310243574 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff686800700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff68815afa0 0x7ff68814eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 -- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6880cc300 msgr2=0x7ff6881f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6880cc300 0x7ff6881f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 -- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff68815afa0 msgr2=0x7ff68814eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff68815afa0 0x7ff68814eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 -- 172.21.15.3:0/2310243574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800094b0 con 0x7ff6880cbf20 2024-09-13T16:50:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff685e00700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6880cbf20 0x7ff68814d670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff67800cc50 tx=0x7ff67800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff67e000700 1 -- 172.21.15.3:0/2310243574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678003e50 con 0x7ff6880cbf20 2024-09-13T16:50:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff67e000700 1 -- 172.21.15.3:0/2310243574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800ce50 con 0x7ff6880cbf20 2024-09-13T16:50:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff67e000700 1 -- 172.21.15.3:0/2310243574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800abb0 con 0x7ff6880cbf20 2024-09-13T16:50:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6881f3600 con 0x7ff6880cbf20 2024-09-13T16:50:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.721+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6881f3d20 con 0x7ff6880cbf20 2024-09-13T16:50:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.725+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68806cc70 con 0x7ff6880cbf20 2024-09-13T16:50:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.725+0000 7ff67e000700 1 -- 172.21.15.3:0/2310243574 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff67800ad10 con 0x7ff6880cbf20 2024-09-13T16:50:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.729+0000 7ff67e000700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff668066730 0x7ff668068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.729+0000 7ff67e000700 1 -- 172.21.15.3:0/2310243574 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff678083fe0 con 0x7ff6880cbf20 2024-09-13T16:50:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.729+0000 7ff685400700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff668066730 0x7ff668068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.729+0000 7ff685400700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff668066730 0x7ff668068bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff674000f60 tx=0x7ff6740079a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.733+0000 7ff67e000700 1 -- 172.21.15.3:0/2310243574 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff6780510b0 con 0x7ff6880cbf20 2024-09-13T16:50:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.925+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6880b4af0 con 0x7ff6880cbf20 2024-09-13T16:50:44.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.925+0000 7ff67e000700 1 -- 172.21.15.3:0/2310243574 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7ff678081790 con 0x7ff6880cbf20 2024-09-13T16:50:44.932 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:44.932 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff668066730 msgr2=0x7ff668068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff668066730 0x7ff668068bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff674000f60 tx=0x7ff6740079a0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6880cbf20 msgr2=0x7ff68814d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6880cbf20 0x7ff68814d670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff67800cc50 tx=0x7ff67800a340 comp rx=0 tx=0).stop 2024-09-13T16:50:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 shutdown_connections 2024-09-13T16:50:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff668066730 0x7ff668068bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff68815afa0 0x7ff68814eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6880cc300 0x7ff6881f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 --2- 172.21.15.3:0/2310243574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6880cbf20 0x7ff68814d670 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 >> 172.21.15.3:0/2310243574 conn(0x7ff688071da0 msgr2=0x7ff68815c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 shutdown_connections 2024-09-13T16:50:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:44.933+0000 7ff687200700 1 -- 172.21.15.3:0/2310243574 wait complete. 2024-09-13T16:50:44.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:50:44.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:50:45.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.141+0000 7fd657000700 1 Processor -- start 2024-09-13T16:50:45.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.141+0000 7fd657000700 1 -- start start 2024-09-13T16:50:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd657000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650158740 0x7fd6500ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd657000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd65015b1c0 0x7fd6500b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd657000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6500b1f30 con 0x7fd6500cc830 2024-09-13T16:50:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd657000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65015b710 con 0x7fd65015b1c0 2024-09-13T16:50:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd657000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65015b9f0 con 0x7fd650158740 2024-09-13T16:50:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd647e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd65015b1c0 0x7fd6500b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd647e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd65015b1c0 0x7fd6500b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52640/0 (socket says 172.21.15.3:52640) 2024-09-13T16:50:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd647e00700 1 -- 172.21.15.3:0/3540240938 learned_addr learned my addr 172.21.15.3:0/3540240938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd64fe00700 1 --2- 172.21.15.3:0/3540240938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650158740 0x7fd6500ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd647e00700 1 -- 172.21.15.3:0/3540240938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650158740 msgr2=0x7fd6500ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd647e00700 1 --2- 172.21.15.3:0/3540240938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650158740 0x7fd6500ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd647e00700 1 -- 172.21.15.3:0/3540240938 >> v1:172.21.15.157:6789/0 conn(0x7fd6500cc830 legacy=0x7fd6500b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd647e00700 1 -- 172.21.15.3:0/3540240938 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65015bcd0 con 0x7fd65015b1c0 2024-09-13T16:50:45.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd647e00700 1 --2- 172.21.15.3:0/3540240938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65015b1c0 0x7fd6500b3250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd638009910 tx=0x7fd638009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be162aaeca42b3c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd64f400700 1 -- 172.21.15.3:0/3540240938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63800ba60 con 0x7fd65015b1c0 2024-09-13T16:50:45.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd64f400700 1 -- 172.21.15.3:0/3540240938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63800bbc0 con 0x7fd65015b1c0 2024-09-13T16:50:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.145+0000 7fd64f400700 1 -- 172.21.15.3:0/3540240938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638009e60 con 0x7fd65015b1c0 2024-09-13T16:50:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 -- 172.21.15.3:0/3540240938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65015b1c0 msgr2=0x7fd6500b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 --2- 172.21.15.3:0/3540240938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65015b1c0 0x7fd6500b3250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd638009910 tx=0x7fd638009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 -- 172.21.15.3:0/3540240938 shutdown_connections 2024-09-13T16:50:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 --2- 172.21.15.3:0/3540240938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65015b1c0 0x7fd6500b3250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 --2- 172.21.15.3:0/3540240938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650158740 0x7fd6500ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 -- 172.21.15.3:0/3540240938 >> 172.21.15.3:0/3540240938 conn(0x7fd65006b280 msgr2=0x7fd65006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 -- 172.21.15.3:0/3540240938 shutdown_connections 2024-09-13T16:50:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 -- 172.21.15.3:0/3540240938 wait complete. 2024-09-13T16:50:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 Processor -- start 2024-09-13T16:50:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.149+0000 7fd657000700 1 -- start start 2024-09-13T16:50:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd657000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6500cc830 0x7fd6501476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd657000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd650158740 0x7fd650070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd657000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650071050 0x7fd65014edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd657000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd650071500 con 0x7fd6500cc830 2024-09-13T16:50:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd657000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65014e970 con 0x7fd650158740 2024-09-13T16:50:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd657000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd650147c00 con 0x7fd650071050 2024-09-13T16:50:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6500cc830 0x7fd6501476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd647e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd650158740 0x7fd650070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6500cc830 0x7fd6501476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52656/0 (socket says 172.21.15.3:52656) 2024-09-13T16:50:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 -- 172.21.15.3:0/2826731431 learned_addr learned my addr 172.21.15.3:0/2826731431 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd655000700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650071050 0x7fd65014edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 -- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650071050 msgr2=0x7fd65014edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650071050 0x7fd65014edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 -- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd650158740 msgr2=0x7fd650070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd650158740 0x7fd650070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 -- 172.21.15.3:0/2826731431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd638009570 con 0x7fd6500cc830 2024-09-13T16:50:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64fe00700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6500cc830 0x7fd6501476c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd64000e760 tx=0x7fd64000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64cc00700 1 -- 172.21.15.3:0/2826731431 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640013610 con 0x7fd6500cc830 2024-09-13T16:50:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64cc00700 1 -- 172.21.15.3:0/2826731431 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd640013770 con 0x7fd6500cc830 2024-09-13T16:50:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd64cc00700 1 -- 172.21.15.3:0/2826731431 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640013ae0 con 0x7fd6500cc830 2024-09-13T16:50:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6500b3910 con 0x7fd6500cc830 2024-09-13T16:50:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.153+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65014fc70 con 0x7fd6500cc830 2024-09-13T16:50:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.157+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65006bd70 con 0x7fd6500cc830 2024-09-13T16:50:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.157+0000 7fd64cc00700 1 -- 172.21.15.3:0/2826731431 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd640013c40 con 0x7fd6500cc830 2024-09-13T16:50:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.161+0000 7fd64cc00700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd62c066730 0x7fd62c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.161+0000 7fd64cc00700 1 -- 172.21.15.3:0/2826731431 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd640084150 con 0x7fd6500cc830 2024-09-13T16:50:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.161+0000 7fd647e00700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd62c066730 0x7fd62c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.161+0000 7fd647e00700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd62c066730 0x7fd62c068bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd638009910 tx=0x7fd638008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.165+0000 7fd64cc00700 1 -- 172.21.15.3:0/2826731431 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd640051130 con 0x7fd6500cc830 2024-09-13T16:50:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.349+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6500af8f0 con 0x7fd6500cc830 2024-09-13T16:50:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.349+0000 7fd64cc00700 1 -- 172.21.15.3:0/2826731431 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd640054750 con 0x7fd6500cc830 2024-09-13T16:50:45.354 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:50:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd62c066730 msgr2=0x7fd62c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd62c066730 0x7fd62c068bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd638009910 tx=0x7fd638008030 comp rx=0 tx=0).stop 2024-09-13T16:50:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6500cc830 msgr2=0x7fd6501476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6500cc830 0x7fd6501476c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd64000e760 tx=0x7fd64000ea70 comp rx=0 tx=0).stop 2024-09-13T16:50:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 shutdown_connections 2024-09-13T16:50:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd62c066730 0x7fd62c068bf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd650071050 0x7fd65014edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd650158740 0x7fd650070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 --2- 172.21.15.3:0/2826731431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6500cc830 0x7fd6501476c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 >> 172.21.15.3:0/2826731431 conn(0x7fd65006b280 msgr2=0x7fd65006d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 shutdown_connections 2024-09-13T16:50:45.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.353+0000 7fd657000700 1 -- 172.21.15.3:0/2826731431 wait complete. 2024-09-13T16:50:45.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:50:45.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.557+0000 7f4977200700 1 Processor -- start 2024-09-13T16:50:45.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.557+0000 7f4977200700 1 -- start start 2024-09-13T16:50:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4977200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f49780cde70 0x7f49780ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4977200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49781650f0 0x7f49780b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4977200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49780ce790 con 0x7f49780cb7c0 2024-09-13T16:50:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4977200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49780b2c70 con 0x7f49780cde70 2024-09-13T16:50:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4977200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49780cc050 con 0x7f49781650f0 2024-09-13T16:50:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f49780cde70 0x7f49780ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f49780cde70 0x7f49780ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52662/0 (socket says 172.21.15.3:52662) 2024-09-13T16:50:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975e00700 1 -- 172.21.15.3:0/3068421880 learned_addr learned my addr 172.21.15.3:0/3068421880 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4976800700 1 --2- 172.21.15.3:0/3068421880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49781650f0 0x7f49780b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975e00700 1 -- 172.21.15.3:0/3068421880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49781650f0 msgr2=0x7f49780b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975e00700 1 --2- 172.21.15.3:0/3068421880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49781650f0 0x7f49780b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975e00700 1 -- 172.21.15.3:0/3068421880 >> v1:172.21.15.3:6789/0 conn(0x7f49780cb7c0 legacy=0x7f49781649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975e00700 1 -- 172.21.15.3:0/3068421880 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49780cc330 con 0x7f49780cde70 2024-09-13T16:50:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4976800700 1 --2- 172.21.15.3:0/3068421880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49781650f0 0x7f49780b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975e00700 1 --2- 172.21.15.3:0/3068421880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cde70 0x7f49780ce250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4968009910 tx=0x7f4968009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ec85e76ce58506 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975400700 1 -- 172.21.15.3:0/3068421880 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496800ba60 con 0x7f49780cde70 2024-09-13T16:50:45.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975400700 1 -- 172.21.15.3:0/3068421880 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496800bbc0 con 0x7f49780cde70 2024-09-13T16:50:45.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.561+0000 7f4975400700 1 -- 172.21.15.3:0/3068421880 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968009e60 con 0x7f49780cde70 2024-09-13T16:50:45.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 -- 172.21.15.3:0/3068421880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cde70 msgr2=0x7f49780ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 --2- 172.21.15.3:0/3068421880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cde70 0x7f49780ce250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4968009910 tx=0x7f4968009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 -- 172.21.15.3:0/3068421880 shutdown_connections 2024-09-13T16:50:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 --2- 172.21.15.3:0/3068421880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49781650f0 0x7f49780b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 --2- 172.21.15.3:0/3068421880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cde70 0x7f49780ce250 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 -- 172.21.15.3:0/3068421880 >> 172.21.15.3:0/3068421880 conn(0x7f49780af620 msgr2=0x7f49780afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 -- 172.21.15.3:0/3068421880 shutdown_connections 2024-09-13T16:50:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 -- 172.21.15.3:0/3068421880 wait complete. 2024-09-13T16:50:45.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.565+0000 7f4977200700 1 Processor -- start 2024-09-13T16:50:45.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 -- start start 2024-09-13T16:50:45.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cb7c0 0x7f4978157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49780cde70 0x7f4978160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49781650f0 0x7f49781596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4978158480 con 0x7f49780cb7c0 2024-09-13T16:50:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497815beb0 con 0x7f49781650f0 2024-09-13T16:50:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497815c190 con 0x7f49780cde70 2024-09-13T16:50:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cb7c0 0x7f4978157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cb7c0 0x7f4978157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52670/0 (socket says 172.21.15.3:52670) 2024-09-13T16:50:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 -- 172.21.15.3:0/1528227451 learned_addr learned my addr 172.21.15.3:0/1528227451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f496d400700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49780cde70 0x7f4978160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4976800700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49781650f0 0x7f49781596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 -- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49780cde70 msgr2=0x7f4978160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49780cde70 0x7f4978160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 -- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49781650f0 msgr2=0x7f49781596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49781650f0 0x7f49781596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 -- 172.21.15.3:0/1528227451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4968009570 con 0x7f49780cb7c0 2024-09-13T16:50:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4976800700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49781650f0 0x7f49781596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4975e00700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cb7c0 0x7f4978157f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49780b1790 tx=0x7f4968004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f496ea00700 1 -- 172.21.15.3:0/1528227451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680036c0 con 0x7f49780cb7c0 2024-09-13T16:50:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f496ea00700 1 -- 172.21.15.3:0/1528227451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496801f070 con 0x7f49780cb7c0 2024-09-13T16:50:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497815c470 con 0x7f49780cb7c0 2024-09-13T16:50:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f496ea00700 1 -- 172.21.15.3:0/1528227451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968003a80 con 0x7f49780cb7c0 2024-09-13T16:50:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.569+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497815ca30 con 0x7f49780cb7c0 2024-09-13T16:50:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.573+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49781f7a30 con 0x7f49780cb7c0 2024-09-13T16:50:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.573+0000 7f496ea00700 1 -- 172.21.15.3:0/1528227451 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4968003be0 con 0x7f49780cb7c0 2024-09-13T16:50:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.573+0000 7f496ea00700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4954066730 0x7f4954068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.573+0000 7f496ea00700 1 -- 172.21.15.3:0/1528227451 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4968021070 con 0x7f49780cb7c0 2024-09-13T16:50:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.577+0000 7f496d400700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4954066730 0x7f4954068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.577+0000 7f496d400700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4954066730 0x7f4954068bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4978164610 tx=0x7f4960009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.581+0000 7f496ea00700 1 -- 172.21.15.3:0/1528227451 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4968052ed0 con 0x7f49780cb7c0 2024-09-13T16:50:45.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.761+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49780ce250 con 0x7f49780cb7c0 2024-09-13T16:50:45.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f496ea00700 1 -- 172.21.15.3:0/1528227451 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4968015020 con 0x7f49780cb7c0 2024-09-13T16:50:45.768 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4954066730 msgr2=0x7f4954068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4954066730 0x7f4954068bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4978164610 tx=0x7f4960009040 comp rx=0 tx=0).stop 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cb7c0 msgr2=0x7f4978157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cb7c0 0x7f4978157f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49780b1790 tx=0x7f4968004770 comp rx=0 tx=0).stop 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 shutdown_connections 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4954066730 0x7f4954068bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49781650f0 0x7f49781596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49780cde70 0x7f4978160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 --2- 172.21.15.3:0/1528227451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49780cb7c0 0x7f4978157f40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 >> 172.21.15.3:0/1528227451 conn(0x7f49780af620 msgr2=0x7f4978070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:45.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 shutdown_connections 2024-09-13T16:50:45.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.765+0000 7f4977200700 1 -- 172.21.15.3:0/1528227451 wait complete. 2024-09-13T16:50:45.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:50:45.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.961+0000 7fb09f400700 1 Processor -- start 2024-09-13T16:50:45.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.961+0000 7fb09f400700 1 -- start start 2024-09-13T16:50:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09f400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098070080 0x7fb0980b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09f400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0980b4d50 con 0x7fb09806f0e0 2024-09-13T16:50:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09f400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0980cfde0 con 0x7fb09815ab10 2024-09-13T16:50:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09f400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09806f550 con 0x7fb098070080 2024-09-13T16:50:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb097e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb09806f0e0 0x7fb09806f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48732/0 (socket says 172.21.15.3:48732) 2024-09-13T16:50:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb097e00700 1 -- 172.21.15.3:0/1479302468 learned_addr learned my addr 172.21.15.3:0/1479302468 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09d400700 1 --2- 172.21.15.3:0/1479302468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098070080 0x7fb0980b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb097400700 1 -- 172.21.15.3:0/1479302468 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642960415 0 0) 0x7fb0980b4d50 con 0x7fb09806f0e0 2024-09-13T16:50:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb097400700 1 -- 172.21.15.3:0/1479302468 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0800035d0 con 0x7fb09806f0e0 2024-09-13T16:50:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09d400700 1 -- 172.21.15.3:0/1479302468 >> v1:172.21.15.157:6789/0 conn(0x7fb09815ab10 legacy=0x7fb0980d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09d400700 1 -- 172.21.15.3:0/1479302468 >> v1:172.21.15.3:6789/0 conn(0x7fb09806f0e0 legacy=0x7fb09806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb097400700 1 -- 172.21.15.3:0/1479302468 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360629248 0 0) 0x7fb0980cfde0 con 0x7fb09815ab10 2024-09-13T16:50:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09d400700 1 -- 172.21.15.3:0/1479302468 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0980b4f80 con 0x7fb098070080 2024-09-13T16:50:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.965+0000 7fb09d400700 1 --2- 172.21.15.3:0/1479302468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098070080 0x7fb0980b4810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb090009dc0 tx=0x7fb09000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1967a7fd655d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb097400700 1 -- 172.21.15.3:0/1479302468 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000e070 con 0x7fb098070080 2024-09-13T16:50:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb097400700 1 -- 172.21.15.3:0/1479302468 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09000bb20 con 0x7fb098070080 2024-09-13T16:50:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb097400700 1 -- 172.21.15.3:0/1479302468 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000bed0 con 0x7fb098070080 2024-09-13T16:50:45.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb09f400700 1 -- 172.21.15.3:0/1479302468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098070080 msgr2=0x7fb0980b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb09f400700 1 --2- 172.21.15.3:0/1479302468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098070080 0x7fb0980b4810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb090009dc0 tx=0x7fb09000b330 comp rx=0 tx=0).stop 2024-09-13T16:50:45.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb09f400700 1 -- 172.21.15.3:0/1479302468 shutdown_connections 2024-09-13T16:50:45.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb09f400700 1 --2- 172.21.15.3:0/1479302468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098070080 0x7fb0980b4810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb09f400700 1 -- 172.21.15.3:0/1479302468 >> 172.21.15.3:0/1479302468 conn(0x7fb0980b2b00 msgr2=0x7fb0980b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:45.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb09f400700 1 -- 172.21.15.3:0/1479302468 shutdown_connections 2024-09-13T16:50:45.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.969+0000 7fb09f400700 1 -- 172.21.15.3:0/1479302468 wait complete. 2024-09-13T16:50:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09f400700 1 Processor -- start 2024-09-13T16:50:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09f400700 1 -- start start 2024-09-13T16:50:45.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb09806f0e0 0x7fb09814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb098070080 0x7fb0980c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09f400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09815ab10 0x7fb0981f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09f400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981f33f0 con 0x7fb09806f0e0 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09f400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981f36b0 con 0x7fb098070080 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09f400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981f3990 con 0x7fb09815ab10 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09ca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb09806f0e0 0x7fb09814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb097e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb098070080 0x7fb0980c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09ca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb09806f0e0 0x7fb09814b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52686/0 (socket says 172.21.15.3:52686) 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09ca00700 1 -- 172.21.15.3:0/671999720 learned_addr learned my addr 172.21.15.3:0/671999720 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.973+0000 7fb09d400700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09815ab10 0x7fb0981f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09ca00700 1 -- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09815ab10 msgr2=0x7fb0981f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09ca00700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09815ab10 0x7fb0981f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09ca00700 1 -- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb098070080 msgr2=0x7fb0980c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09ca00700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb098070080 0x7fb0980c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09ca00700 1 -- 172.21.15.3:0/671999720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090009a70 con 0x7fb09806f0e0 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb097e00700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb098070080 0x7fb0980c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09ca00700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb09806f0e0 0x7fb09814b790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c450 tx=0x7fb08c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb094c00700 1 -- 172.21.15.3:0/671999720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0054e0 con 0x7fb09806f0e0 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb094c00700 1 -- 172.21.15.3:0/671999720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c00ce60 con 0x7fb09806f0e0 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb094c00700 1 -- 172.21.15.3:0/671999720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0038e0 con 0x7fb09806f0e0 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0981f3c70 con 0x7fb09806f0e0 2024-09-13T16:50:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0981f4230 con 0x7fb09806f0e0 2024-09-13T16:50:45.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.977+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0980b5f00 con 0x7fb09806f0e0 2024-09-13T16:50:45.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.981+0000 7fb094c00700 1 -- 172.21.15.3:0/671999720 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb08c003a40 con 0x7fb09806f0e0 2024-09-13T16:50:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.981+0000 7fb094c00700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb078066760 0x7fb078068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.981+0000 7fb094c00700 1 -- 172.21.15.3:0/671999720 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb08c083770 con 0x7fb09806f0e0 2024-09-13T16:50:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.981+0000 7fb097e00700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb078066760 0x7fb078068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.985+0000 7fb097e00700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb078066760 0x7fb078068c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0980b2310 tx=0x7fb084007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:45.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:45.989+0000 7fb094c00700 1 -- 172.21.15.3:0/671999720 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb08c0507e0 con 0x7fb09806f0e0 2024-09-13T16:50:46.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.177+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0980b4810 con 0x7fb09806f0e0 2024-09-13T16:50:46.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.177+0000 7fb094c00700 1 -- 172.21.15.3:0/671999720 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb08c017020 con 0x7fb09806f0e0 2024-09-13T16:50:46.181 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:50:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.177+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb078066760 msgr2=0x7fb078068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.177+0000 7fb09f400700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb078066760 0x7fb078068c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0980b2310 tx=0x7fb084007de0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.177+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb09806f0e0 msgr2=0x7fb09814b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.177+0000 7fb09f400700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb09806f0e0 0x7fb09814b790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c450 tx=0x7fb08c00c760 comp rx=0 tx=0).stop 2024-09-13T16:50:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.181+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 shutdown_connections 2024-09-13T16:50:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.181+0000 7fb09f400700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb078066760 0x7fb078068c20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.181+0000 7fb09f400700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09815ab10 0x7fb0981f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.181+0000 7fb09f400700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb098070080 0x7fb0980c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.181+0000 7fb09f400700 1 --2- 172.21.15.3:0/671999720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb09806f0e0 0x7fb09814b790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.181+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 >> 172.21.15.3:0/671999720 conn(0x7fb0980b2b00 msgr2=0x7fb0980d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.181+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 shutdown_connections 2024-09-13T16:50:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.181+0000 7fb09f400700 1 -- 172.21.15.3:0/671999720 wait complete. 2024-09-13T16:50:46.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:50:46.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.381+0000 7fd510c00700 1 Processor -- start 2024-09-13T16:50:46.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.381+0000 7fd510c00700 1 -- start start 2024-09-13T16:50:46.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd510c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd50c0cde70 0x7fd50c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd510c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c0ce790 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd510c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c0cc050 con 0x7fd50c1650f0 2024-09-13T16:50:46.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd510c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0b2c70 con 0x7fd50c0cde70 2024-09-13T16:50:46.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50b400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd50c0cde70 0x7fd50c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50aa00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd50c0cb7c0 0x7fd50c1649e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48738/0 (socket says 172.21.15.3:48738) 2024-09-13T16:50:46.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50aa00700 1 -- 172.21.15.3:0/1070985991 learned_addr learned my addr 172.21.15.3:0/1070985991 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:46.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50b400700 1 -- 172.21.15.3:0/1070985991 >> v1:172.21.15.157:6789/0 conn(0x7fd50c1650f0 legacy=0x7fd50c0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50b400700 1 -- 172.21.15.3:0/1070985991 >> v1:172.21.15.3:6789/0 conn(0x7fd50c0cb7c0 legacy=0x7fd50c1649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50b400700 1 -- 172.21.15.3:0/1070985991 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0cc330 con 0x7fd50c0cde70 2024-09-13T16:50:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50b400700 1 --2- 172.21.15.3:0/1070985991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cde70 0x7fd50c0ce250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009850 tx=0x7fd4fc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f909f9a697a62f37 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50a000700 1 -- 172.21.15.3:0/1070985991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc011830 con 0x7fd50c0cde70 2024-09-13T16:50:46.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50a000700 1 -- 172.21.15.3:0/1070985991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc011990 con 0x7fd50c0cde70 2024-09-13T16:50:46.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.385+0000 7fd50a000700 1 -- 172.21.15.3:0/1070985991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc011d20 con 0x7fd50c0cde70 2024-09-13T16:50:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.389+0000 7fd510c00700 1 -- 172.21.15.3:0/1070985991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cde70 msgr2=0x7fd50c0ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.389+0000 7fd510c00700 1 --2- 172.21.15.3:0/1070985991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cde70 0x7fd50c0ce250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009850 tx=0x7fd4fc009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.389+0000 7fd510c00700 1 -- 172.21.15.3:0/1070985991 shutdown_connections 2024-09-13T16:50:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.389+0000 7fd510c00700 1 --2- 172.21.15.3:0/1070985991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cde70 0x7fd50c0ce250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.389+0000 7fd510c00700 1 -- 172.21.15.3:0/1070985991 >> 172.21.15.3:0/1070985991 conn(0x7fd50c0af620 msgr2=0x7fd50c0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:46.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.389+0000 7fd510c00700 1 -- 172.21.15.3:0/1070985991 shutdown_connections 2024-09-13T16:50:46.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.389+0000 7fd510c00700 1 -- 172.21.15.3:0/1070985991 wait complete. 2024-09-13T16:50:46.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd510c00700 1 Processor -- start 2024-09-13T16:50:46.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd510c00700 1 -- start start 2024-09-13T16:50:46.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd510c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cb7c0 0x7fd50c06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd510c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50c0cde70 0x7fd50c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd510c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd50c1650f0 0x7fd50c152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd510c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c163340 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd510c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c1646a0 con 0x7fd50c0cde70 2024-09-13T16:50:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd510c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c163590 con 0x7fd50c1650f0 2024-09-13T16:50:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cb7c0 0x7fd50c06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cb7c0 0x7fd50c06cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52704/0 (socket says 172.21.15.3:52704) 2024-09-13T16:50:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50b400700 1 -- 172.21.15.3:0/2137764267 learned_addr learned my addr 172.21.15.3:0/2137764267 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50aa00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50c0cde70 0x7fd50c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50be00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd50c1650f0 0x7fd50c152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50b400700 1 -- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd50c1650f0 msgr2=0x7fd50c152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50b400700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd50c1650f0 0x7fd50c152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50b400700 1 -- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50c0cde70 msgr2=0x7fd50c1500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50b400700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50c0cde70 0x7fd50c1500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50b400700 1 -- 172.21.15.3:0/2137764267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0094b0 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.393+0000 7fd50be00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd50c1650f0 0x7fd50c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.397+0000 7fd50b400700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cb7c0 0x7fd50c06cde0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0ce920 tx=0x7fd4fc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.397+0000 7fd503400700 1 -- 172.21.15.3:0/2137764267 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0036c0 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.397+0000 7fd503400700 1 -- 172.21.15.3:0/2137764267 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc003820 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.397+0000 7fd503400700 1 -- 172.21.15.3:0/2137764267 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc003bd0 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.397+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c163870 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.397+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c06c8c0 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.397+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c06c440 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.401+0000 7fd503400700 1 -- 172.21.15.3:0/2137764267 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4fc003d30 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.401+0000 7fd503400700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4ec066730 0x7fd4ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.401+0000 7fd503400700 1 -- 172.21.15.3:0/2137764267 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd4fc021030 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.401+0000 7fd50aa00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4ec066730 0x7fd4ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.405+0000 7fd50aa00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4ec066730 0x7fd4ec068bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005930 tx=0x7fd4f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.409+0000 7fd503400700 1 -- 172.21.15.3:0/2137764267 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd50c06c440 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.593+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd50c06c440 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.593+0000 7fd503400700 1 -- 172.21.15.3:0/2137764267 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd50c06c440 con 0x7fd50c0cb7c0 2024-09-13T16:50:46.599 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:50:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4ec066730 msgr2=0x7fd4ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4ec066730 0x7fd4ec068bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005930 tx=0x7fd4f8009040 comp rx=0 tx=0).stop 2024-09-13T16:50:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cb7c0 msgr2=0x7fd50c06cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cb7c0 0x7fd50c06cde0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0ce920 tx=0x7fd4fc004770 comp rx=0 tx=0).stop 2024-09-13T16:50:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 shutdown_connections 2024-09-13T16:50:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd4ec066730 0x7fd4ec068bf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd50c1650f0 0x7fd50c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50c0cde70 0x7fd50c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 --2- 172.21.15.3:0/2137764267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50c0cb7c0 0x7fd50c06cde0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 >> 172.21.15.3:0/2137764267 conn(0x7fd50c0af620 msgr2=0x7fd50c15ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 shutdown_connections 2024-09-13T16:50:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.601+0000 7fd510c00700 1 -- 172.21.15.3:0/2137764267 wait complete. 2024-09-13T16:50:46.628 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:50:46.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.797+0000 7f3b44a00700 1 Processor -- start 2024-09-13T16:50:46.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.797+0000 7f3b44a00700 1 -- start start 2024-09-13T16:50:46.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b44a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b400d03e0 0x7f3b400b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b44a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b4006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b44a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b40165030 con 0x7f3b4006fe20 2024-09-13T16:50:46.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b44a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b400702d0 con 0x7f3b400d03e0 2024-09-13T16:50:46.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b44a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40165310 con 0x7f3b400b4410 2024-09-13T16:50:46.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3f400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b400d03e0 0x7f3b400b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3f400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b400d03e0 0x7f3b400b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52716/0 (socket says 172.21.15.3:52716) 2024-09-13T16:50:46.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3f400700 1 -- 172.21.15.3:0/1551581962 learned_addr learned my addr 172.21.15.3:0/1551581962 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:46.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3ea00700 1 --2- 172.21.15.3:0/1551581962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b4006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3f400700 1 -- 172.21.15.3:0/1551581962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 msgr2=0x7f3b4006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:46.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3f400700 1 --2- 172.21.15.3:0/1551581962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b4006f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3f400700 1 -- 172.21.15.3:0/1551581962 >> v1:172.21.15.3:6789/0 conn(0x7f3b4006fe20 legacy=0x7f3b40146880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3f400700 1 -- 172.21.15.3:0/1551581962 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b401655f0 con 0x7f3b400d03e0 2024-09-13T16:50:46.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3ea00700 1 --2- 172.21.15.3:0/1551581962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b4006f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:46.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3f400700 1 --2- 172.21.15.3:0/1551581962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b400d03e0 0x7f3b400b3ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009910 tx=0x7f3b30009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a03dc7cdf193b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3e000700 1 -- 172.21.15.3:0/1551581962 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000ba60 con 0x7f3b400d03e0 2024-09-13T16:50:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3e000700 1 -- 172.21.15.3:0/1551581962 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3000bbc0 con 0x7f3b400d03e0 2024-09-13T16:50:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.801+0000 7f3b3e000700 1 -- 172.21.15.3:0/1551581962 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30009e60 con 0x7f3b400d03e0 2024-09-13T16:50:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 -- 172.21.15.3:0/1551581962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b400d03e0 msgr2=0x7f3b400b3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 --2- 172.21.15.3:0/1551581962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b400d03e0 0x7f3b400b3ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009910 tx=0x7f3b30009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 -- 172.21.15.3:0/1551581962 shutdown_connections 2024-09-13T16:50:46.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 --2- 172.21.15.3:0/1551581962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b4006f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 --2- 172.21.15.3:0/1551581962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b400d03e0 0x7f3b400b3ed0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 -- 172.21.15.3:0/1551581962 >> 172.21.15.3:0/1551581962 conn(0x7f3b400b2910 msgr2=0x7f3b400b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:46.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 -- 172.21.15.3:0/1551581962 shutdown_connections 2024-09-13T16:50:46.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 -- 172.21.15.3:0/1551581962 wait complete. 2024-09-13T16:50:46.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 Processor -- start 2024-09-13T16:50:46.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.805+0000 7f3b44a00700 1 -- start start 2024-09-13T16:50:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b44a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4006fe20 0x7f3b40159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b44a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b401494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b44a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b400d03e0 0x7f3b401fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b44a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4015c5d0 con 0x7f3b4006fe20 2024-09-13T16:50:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b44a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4015b530 con 0x7f3b400d03e0 2024-09-13T16:50:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b44a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4015b810 con 0x7f3b400b4410 2024-09-13T16:50:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4006fe20 0x7f3b40159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4006fe20 0x7f3b40159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52720/0 (socket says 172.21.15.3:52720) 2024-09-13T16:50:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 -- 172.21.15.3:0/2141799820 learned_addr learned my addr 172.21.15.3:0/2141799820 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3ea00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b401494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3fe00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b400d03e0 0x7f3b401fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 -- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 msgr2=0x7f3b401494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b401494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 -- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b400d03e0 msgr2=0x7f3b401fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b400d03e0 0x7f3b401fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 -- 172.21.15.3:0/2141799820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b30009570 con 0x7f3b4006fe20 2024-09-13T16:50:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3fe00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b400d03e0 0x7f3b401fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b3f400700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4006fe20 0x7f3b40159a30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b400b3980 tx=0x7f3b30004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b37400700 1 -- 172.21.15.3:0/2141799820 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004510 con 0x7f3b4006fe20 2024-09-13T16:50:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b37400700 1 -- 172.21.15.3:0/2141799820 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3001f070 con 0x7f3b4006fe20 2024-09-13T16:50:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.809+0000 7f3b37400700 1 -- 172.21.15.3:0/2141799820 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30003900 con 0x7f3b4006fe20 2024-09-13T16:50:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.813+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4015baf0 con 0x7f3b4006fe20 2024-09-13T16:50:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.813+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4015c0b0 con 0x7f3b4006fe20 2024-09-13T16:50:46.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.813+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4006f8e0 con 0x7f3b4006fe20 2024-09-13T16:50:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.813+0000 7f3b37400700 1 -- 172.21.15.3:0/2141799820 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3b30003a60 con 0x7f3b4006fe20 2024-09-13T16:50:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.817+0000 7f3b37400700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b20066730 0x7f3b20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.817+0000 7f3b37400700 1 -- 172.21.15.3:0/2141799820 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3b30021070 con 0x7f3b4006fe20 2024-09-13T16:50:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.817+0000 7f3b3ea00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b20066730 0x7f3b20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.817+0000 7f3b3ea00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b20066730 0x7f3b20068bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b401f8310 tx=0x7f3b2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:46.821+0000 7f3b37400700 1 -- 172.21.15.3:0/2141799820 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3b30052cd0 con 0x7f3b4006fe20 2024-09-13T16:50:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.025+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3b400b1ff0 con 0x7f3b4006fe20 2024-09-13T16:50:47.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.025+0000 7f3b37400700 1 -- 172.21.15.3:0/2141799820 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3b30015080 con 0x7f3b4006fe20 2024-09-13T16:50:47.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b20066730 msgr2=0x7f3b20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:47.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b20066730 0x7f3b20068bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b401f8310 tx=0x7f3b2c008040 comp rx=0 tx=0).stop 2024-09-13T16:50:47.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4006fe20 msgr2=0x7f3b40159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4006fe20 0x7f3b40159a30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b400b3980 tx=0x7f3b30004770 comp rx=0 tx=0).stop 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 shutdown_connections 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b20066730 0x7f3b20068bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b400d03e0 0x7f3b401fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400b4410 0x7f3b401494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 --2- 172.21.15.3:0/2141799820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4006fe20 0x7f3b40159a30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 >> 172.21.15.3:0/2141799820 conn(0x7f3b400b2910 msgr2=0x7f3b401613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 shutdown_connections 2024-09-13T16:50:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:47.029+0000 7f3b44a00700 1 -- 172.21.15.3:0/2141799820 wait complete. 2024-09-13T16:50:47.037 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-13T16:50:47.059 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-13T16:50:47.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:47.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:50:47.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[32222]: fuse finished with error 0 and tester_r 0 2024-09-13T16:50:47.110 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:50:53.212 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:50:53.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:50:53.212 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-13T16:50:53.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:53.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.465+0000 7f6a01000700 1 Processor -- start 2024-09-13T16:50:53.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.465+0000 7f6a01000700 1 -- start start 2024-09-13T16:50:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f6a01000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc159400 0x7f69fc0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f6a01000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f69fc0cdaf0 0x7f69fc070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f6a01000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc0b3e50 con 0x7f69fc159020 2024-09-13T16:50:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f6a01000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc0b4130 con 0x7f69fc0cdaf0 2024-09-13T16:50:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f6a01000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0ccdb0 con 0x7f69fc159400 2024-09-13T16:50:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f69f1400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc159400 0x7f69fc0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f69f9e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f69fc159020 0x7f69fc0b27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:59760/0 (socket says 172.21.15.3:59760) 2024-09-13T16:50:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f69f9e00700 1 -- 172.21.15.3:0/4051787437 learned_addr learned my addr 172.21.15.3:0/4051787437 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:53.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.469+0000 7f69fa800700 1 --2- 172.21.15.3:0/4051787437 >> v2:172.21.15.157:3300/0 conn(0x7f69fc0cdaf0 0x7f69fc070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f9400700 1 -- 172.21.15.3:0/4051787437 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308017073 0 0) 0x7f69fc0b3e50 con 0x7f69fc159020 2024-09-13T16:50:53.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f9400700 1 -- 172.21.15.3:0/4051787437 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e00035d0 con 0x7f69fc159020 2024-09-13T16:50:53.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f1400700 1 -- 172.21.15.3:0/4051787437 >> v2:172.21.15.157:3300/0 conn(0x7f69fc0cdaf0 msgr2=0x7f69fc070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:53.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f1400700 1 --2- 172.21.15.3:0/4051787437 >> v2:172.21.15.157:3300/0 conn(0x7f69fc0cdaf0 0x7f69fc070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f1400700 1 -- 172.21.15.3:0/4051787437 >> v1:172.21.15.157:6789/0 conn(0x7f69fc159020 legacy=0x7f69fc0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f1400700 1 -- 172.21.15.3:0/4051787437 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc0cd090 con 0x7f69fc159400 2024-09-13T16:50:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f1400700 1 --2- 172.21.15.3:0/4051787437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc159400 0x7f69fc0cd5b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009850 tx=0x7f69e4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7fba8f07635b786 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f9400700 1 -- 172.21.15.3:0/4051787437 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4011830 con 0x7f69fc159400 2024-09-13T16:50:53.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f9400700 1 -- 172.21.15.3:0/4051787437 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e4011990 con 0x7f69fc159400 2024-09-13T16:50:53.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f69f9400700 1 -- 172.21.15.3:0/4051787437 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4011d40 con 0x7f69fc159400 2024-09-13T16:50:53.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f6a01000700 1 -- 172.21.15.3:0/4051787437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc159400 msgr2=0x7f69fc0cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f6a01000700 1 --2- 172.21.15.3:0/4051787437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc159400 0x7f69fc0cd5b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009850 tx=0x7f69e4009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f6a01000700 1 -- 172.21.15.3:0/4051787437 shutdown_connections 2024-09-13T16:50:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f6a01000700 1 --2- 172.21.15.3:0/4051787437 >> v2:172.21.15.157:3300/0 conn(0x7f69fc0cdaf0 0x7f69fc070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f6a01000700 1 --2- 172.21.15.3:0/4051787437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc159400 0x7f69fc0cd5b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f6a01000700 1 -- 172.21.15.3:0/4051787437 >> 172.21.15.3:0/4051787437 conn(0x7f69fc06a390 msgr2=0x7f69fc06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f6a01000700 1 -- 172.21.15.3:0/4051787437 shutdown_connections 2024-09-13T16:50:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.473+0000 7f6a01000700 1 -- 172.21.15.3:0/4051787437 wait complete. 2024-09-13T16:50:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f6a01000700 1 Processor -- start 2024-09-13T16:50:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f6a01000700 1 -- start start 2024-09-13T16:50:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f6a01000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69fc0cdaf0 0x7f69fc070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f6a01000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159020 0x7f69fc14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f6a01000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc157ac0 0x7f69fc14c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f6a01000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc14fb60 con 0x7f69fc159020 2024-09-13T16:50:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f6a01000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc14fe40 con 0x7f69fc0cdaf0 2024-09-13T16:50:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f6a01000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc150120 con 0x7f69fc157ac0 2024-09-13T16:50:53.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f69f1400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159020 0x7f69fc14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f69f1400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159020 0x7f69fc14bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52736/0 (socket says 172.21.15.3:52736) 2024-09-13T16:50:53.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.477+0000 7f69f1400700 1 -- 172.21.15.3:0/1670208862 learned_addr learned my addr 172.21.15.3:0/1670208862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:53.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69fa800700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc157ac0 0x7f69fc14c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f9e00700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69fc0cdaf0 0x7f69fc070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f1400700 1 -- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc157ac0 msgr2=0x7f69fc14c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f1400700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc157ac0 0x7f69fc14c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f1400700 1 -- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69fc0cdaf0 msgr2=0x7f69fc070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f1400700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69fc0cdaf0 0x7f69fc070fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f1400700 1 -- 172.21.15.3:0/1670208862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e40094b0 con 0x7f69fc159020 2024-09-13T16:50:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f9e00700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69fc0cdaf0 0x7f69fc070fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:53.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f1400700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159020 0x7f69fc14bfc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f69e4014040 tx=0x7f69e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f2a00700 1 -- 172.21.15.3:0/1670208862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4004590 con 0x7f69fc159020 2024-09-13T16:50:53.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f2a00700 1 -- 172.21.15.3:0/1670208862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e401e070 con 0x7f69fc159020 2024-09-13T16:50:53.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69fc150400 con 0x7f69fc159020 2024-09-13T16:50:53.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f69f2a00700 1 -- 172.21.15.3:0/1670208862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4003900 con 0x7f69fc159020 2024-09-13T16:50:53.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69fc071510 con 0x7f69fc159020 2024-09-13T16:50:53.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.481+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69fc071700 con 0x7f69fc159020 2024-09-13T16:50:53.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.485+0000 7f69f2a00700 1 -- 172.21.15.3:0/1670208862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69e4003a60 con 0x7f69fc159020 2024-09-13T16:50:53.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.485+0000 7f69f2a00700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f69d8066730 0x7f69d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.485+0000 7f69f2a00700 1 -- 172.21.15.3:0/1670208862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69e4020070 con 0x7f69fc159020 2024-09-13T16:50:53.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.485+0000 7f69f9e00700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f69d8066730 0x7f69d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.489+0000 7f69f9e00700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f69d8066730 0x7f69d8068bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f69ec005930 tx=0x7f69ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.493+0000 7f69f2a00700 1 -- 172.21.15.3:0/1670208862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69e40528d0 con 0x7f69fc159020 2024-09-13T16:50:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.685+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69fc150590 con 0x7f69fc159020 2024-09-13T16:50:53.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.689+0000 7f69f2a00700 1 -- 172.21.15.3:0/1670208862 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7f69e4006ce0 con 0x7f69fc159020 2024-09-13T16:50:53.696 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:53.696 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f69d8066730 msgr2=0x7f69d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f69d8066730 0x7f69d8068bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f69ec005930 tx=0x7f69ec009040 comp rx=0 tx=0).stop 2024-09-13T16:50:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159020 msgr2=0x7f69fc14bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159020 0x7f69fc14bfc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f69e4014040 tx=0x7f69e4004770 comp rx=0 tx=0).stop 2024-09-13T16:50:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 shutdown_connections 2024-09-13T16:50:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f69d8066730 0x7f69d8068bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69fc157ac0 0x7f69fc14c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159020 0x7f69fc14bfc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 --2- 172.21.15.3:0/1670208862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69fc0cdaf0 0x7f69fc070fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 >> 172.21.15.3:0/1670208862 conn(0x7f69fc06a390 msgr2=0x7f69fc0ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 shutdown_connections 2024-09-13T16:50:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.693+0000 7f6a01000700 1 -- 172.21.15.3:0/1670208862 wait complete. 2024-09-13T16:50:53.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.885+0000 7fdab5800700 1 Processor -- start 2024-09-13T16:50:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdab5800700 1 -- start start 2024-09-13T16:50:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdab5800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdab015ab10 0x7fdab006e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdab5800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab006fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdab5800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab00cf220 con 0x7fdab00b6140 2024-09-13T16:50:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdab5800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab006e5d0 con 0x7fdab015ab10 2024-09-13T16:50:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdab5800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab0070400 con 0x7fdab006fab0 2024-09-13T16:50:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdaaea00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdab015ab10 0x7fdab006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdaae000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdab00b6140 0x7fdab00d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39386/0 (socket says 172.21.15.3:39386) 2024-09-13T16:50:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdaae000700 1 -- 172.21.15.3:0/2971662573 learned_addr learned my addr 172.21.15.3:0/2971662573 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.889+0000 7fdaaf400700 1 --2- 172.21.15.3:0/2971662573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab006fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaad600700 1 -- 172.21.15.3:0/2971662573 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255035775 0 0) 0x7fdab00cf220 con 0x7fdab00b6140 2024-09-13T16:50:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaad600700 1 -- 172.21.15.3:0/2971662573 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda940035d0 con 0x7fdab00b6140 2024-09-13T16:50:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaaea00700 1 -- 172.21.15.3:0/2971662573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 msgr2=0x7fdab006fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaaea00700 1 --2- 172.21.15.3:0/2971662573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab006fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaaea00700 1 -- 172.21.15.3:0/2971662573 >> v1:172.21.15.3:6789/0 conn(0x7fdab00b6140 legacy=0x7fdab00d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaaea00700 1 -- 172.21.15.3:0/2971662573 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab00cf4c0 con 0x7fdab015ab10 2024-09-13T16:50:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaaf400700 1 --2- 172.21.15.3:0/2971662573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab006fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaaea00700 1 --2- 172.21.15.3:0/2971662573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab015ab10 0x7fdab006e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009850 tx=0x7fdaa0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c205f31f21b938 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaad600700 1 -- 172.21.15.3:0/2971662573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0011830 con 0x7fdab015ab10 2024-09-13T16:50:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaad600700 1 -- 172.21.15.3:0/2971662573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa0011990 con 0x7fdab015ab10 2024-09-13T16:50:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdaad600700 1 -- 172.21.15.3:0/2971662573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0011d00 con 0x7fdab015ab10 2024-09-13T16:50:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdab5800700 1 -- 172.21.15.3:0/2971662573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab015ab10 msgr2=0x7fdab006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdab5800700 1 --2- 172.21.15.3:0/2971662573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab015ab10 0x7fdab006e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009850 tx=0x7fdaa0009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdab5800700 1 -- 172.21.15.3:0/2971662573 shutdown_connections 2024-09-13T16:50:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdab5800700 1 --2- 172.21.15.3:0/2971662573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab006fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdab5800700 1 --2- 172.21.15.3:0/2971662573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab015ab10 0x7fdab006e090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdab5800700 1 -- 172.21.15.3:0/2971662573 >> 172.21.15.3:0/2971662573 conn(0x7fdab00d0e30 msgr2=0x7fdab00d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdab5800700 1 -- 172.21.15.3:0/2971662573 shutdown_connections 2024-09-13T16:50:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.893+0000 7fdab5800700 1 -- 172.21.15.3:0/2971662573 wait complete. 2024-09-13T16:50:53.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdab5800700 1 Processor -- start 2024-09-13T16:50:53.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdab5800700 1 -- start start 2024-09-13T16:50:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdab5800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab0159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdab5800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab00b6140 0x7fdab01f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdab5800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab015ab10 0x7fdab014ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdab5800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab0149c00 con 0x7fdab00b6140 2024-09-13T16:50:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdab5800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab015db60 con 0x7fdab015ab10 2024-09-13T16:50:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdab5800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab0148320 con 0x7fdab006fab0 2024-09-13T16:50:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdaae000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab00b6140 0x7fdab01f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdaae000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab00b6140 0x7fdab01f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54326/0 (socket says 172.21.15.3:54326) 2024-09-13T16:50:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdaae000700 1 -- 172.21.15.3:0/3419881928 learned_addr learned my addr 172.21.15.3:0/3419881928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.897+0000 7fdaaea00700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab0159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaaf400700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab015ab10 0x7fdab014ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaae000700 1 -- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 msgr2=0x7fdab0159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaae000700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab0159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaae000700 1 -- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab015ab10 msgr2=0x7fdab014ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaae000700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab015ab10 0x7fdab014ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaae000700 1 -- 172.21.15.3:0/3419881928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa00094b0 con 0x7fdab00b6140 2024-09-13T16:50:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaaea00700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab0159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaae000700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab00b6140 0x7fdab01f0f70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda9c001200 tx=0x7fda9c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaa6a00700 1 -- 172.21.15.3:0/3419881928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c005d90 con 0x7fdab00b6140 2024-09-13T16:50:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaa6a00700 1 -- 172.21.15.3:0/3419881928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c0070d0 con 0x7fdab00b6140 2024-09-13T16:50:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab0148ff0 con 0x7fdab00b6140 2024-09-13T16:50:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdaa6a00700 1 -- 172.21.15.3:0/3419881928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c007480 con 0x7fdab00b6140 2024-09-13T16:50:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab01495b0 con 0x7fdab00b6140 2024-09-13T16:50:53.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.901+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab006fec0 con 0x7fdab00b6140 2024-09-13T16:50:53.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.905+0000 7fdaa6a00700 1 -- 172.21.15.3:0/3419881928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda9c0075e0 con 0x7fdab00b6140 2024-09-13T16:50:53.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.905+0000 7fdaa6a00700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fda90066760 0x7fda90068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.905+0000 7fdaa6a00700 1 -- 172.21.15.3:0/3419881928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fda9c0838e0 con 0x7fdab00b6140 2024-09-13T16:50:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.905+0000 7fdaaea00700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fda90066760 0x7fda90068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.909+0000 7fdaaea00700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fda90066760 0x7fda90068c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00090b0 tx=0x7fdaa0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:53.913+0000 7fdaa6a00700 1 -- 172.21.15.3:0/3419881928 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fda9c050960 con 0x7fdab00b6140 2024-09-13T16:50:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.105+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdab015e5c0 con 0x7fdab00b6140 2024-09-13T16:50:54.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.105+0000 7fdaa6a00700 1 -- 172.21.15.3:0/3419881928 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7fda9c053f80 con 0x7fdab00b6140 2024-09-13T16:50:54.111 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:54.111 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:54.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fda90066760 msgr2=0x7fda90068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fda90066760 0x7fda90068c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00090b0 tx=0x7fdaa0008030 comp rx=0 tx=0).stop 2024-09-13T16:50:54.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab00b6140 msgr2=0x7fdab01f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab00b6140 0x7fdab01f0f70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda9c001200 tx=0x7fda9c00ca90 comp rx=0 tx=0).stop 2024-09-13T16:50:54.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 shutdown_connections 2024-09-13T16:50:54.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fda90066760 0x7fda90068c20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab015ab10 0x7fdab014ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab00b6140 0x7fdab01f0f70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 --2- 172.21.15.3:0/3419881928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdab006fab0 0x7fdab0159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 >> 172.21.15.3:0/3419881928 conn(0x7fdab00d0e30 msgr2=0x7fdab00b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:54.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 shutdown_connections 2024-09-13T16:50:54.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.109+0000 7fdab5800700 1 -- 172.21.15.3:0/3419881928 wait complete. 2024-09-13T16:50:54.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T16:50:54.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.309+0000 7ff06a800700 1 Processor -- start 2024-09-13T16:50:54.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.309+0000 7ff06a800700 1 -- start start 2024-09-13T16:50:54.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff06a800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0640cde70 0x7ff0640ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff06a800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cb7c0 0x7ff0641649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff06a800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640ce790 con 0x7ff064164f20 2024-09-13T16:50:54.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff06a800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640b2c70 con 0x7ff0640cde70 2024-09-13T16:50:54.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff06a800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641653b0 con 0x7ff0640cb7c0 2024-09-13T16:50:54.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff069e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff064164f20 0x7ff064161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39388/0 (socket says 172.21.15.3:39388) 2024-09-13T16:50:54.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff069e00700 1 -- 172.21.15.3:0/3859707290 learned_addr learned my addr 172.21.15.3:0/3859707290 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:54.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff069400700 1 --2- 172.21.15.3:0/3859707290 >> v2:172.21.15.157:3300/0 conn(0x7ff0640cde70 0x7ff0640ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff068a00700 1 --2- 172.21.15.3:0/3859707290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cb7c0 0x7ff0641649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff063e00700 1 -- 172.21.15.3:0/3859707290 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922753487 0 0) 0x7ff0640ce790 con 0x7ff064164f20 2024-09-13T16:50:54.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff063e00700 1 -- 172.21.15.3:0/3859707290 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff04c0035d0 con 0x7ff064164f20 2024-09-13T16:50:54.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff069400700 1 -- 172.21.15.3:0/3859707290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cb7c0 msgr2=0x7ff0641649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:54.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff069400700 1 --2- 172.21.15.3:0/3859707290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cb7c0 0x7ff0641649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff069400700 1 -- 172.21.15.3:0/3859707290 >> v1:172.21.15.3:6789/0 conn(0x7ff064164f20 legacy=0x7ff064161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff069400700 1 -- 172.21.15.3:0/3859707290 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640cc050 con 0x7ff0640cde70 2024-09-13T16:50:54.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff069400700 1 --2- 172.21.15.3:0/3859707290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cde70 0x7ff0640ce250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff058009850 tx=0x7ff058009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166011f4a0f5a6c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:54.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.313+0000 7ff063e00700 1 -- 172.21.15.3:0/3859707290 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058011830 con 0x7ff0640cde70 2024-09-13T16:50:54.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff063e00700 1 -- 172.21.15.3:0/3859707290 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff058011990 con 0x7ff0640cde70 2024-09-13T16:50:54.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff063e00700 1 -- 172.21.15.3:0/3859707290 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058011d40 con 0x7ff0640cde70 2024-09-13T16:50:54.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff06a800700 1 -- 172.21.15.3:0/3859707290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cde70 msgr2=0x7ff0640ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff06a800700 1 --2- 172.21.15.3:0/3859707290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cde70 0x7ff0640ce250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff058009850 tx=0x7ff058009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:54.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff06a800700 1 -- 172.21.15.3:0/3859707290 shutdown_connections 2024-09-13T16:50:54.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff06a800700 1 --2- 172.21.15.3:0/3859707290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cb7c0 0x7ff0641649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff06a800700 1 --2- 172.21.15.3:0/3859707290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cde70 0x7ff0640ce250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff06a800700 1 -- 172.21.15.3:0/3859707290 >> 172.21.15.3:0/3859707290 conn(0x7ff0640af620 msgr2=0x7ff0640afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:54.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff06a800700 1 -- 172.21.15.3:0/3859707290 shutdown_connections 2024-09-13T16:50:54.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.317+0000 7ff06a800700 1 -- 172.21.15.3:0/3859707290 wait complete. 2024-09-13T16:50:54.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff06a800700 1 Processor -- start 2024-09-13T16:50:54.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff06a800700 1 -- start start 2024-09-13T16:50:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff06a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cb7c0 0x7ff06415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff06a800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cde70 0x7ff06415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff06a800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff064164f20 0x7ff06415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff06a800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06415cc80 con 0x7ff064164f20 2024-09-13T16:50:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff06a800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06415cf60 con 0x7ff0640cb7c0 2024-09-13T16:50:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff06a800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06415d240 con 0x7ff0640cde70 2024-09-13T16:50:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff069e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff064164f20 0x7ff06415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff069400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cb7c0 0x7ff06415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff069e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff064164f20 0x7ff06415eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54334/0 (socket says 172.21.15.3:54334) 2024-09-13T16:50:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff069e00700 1 -- 172.21.15.3:0/750080389 learned_addr learned my addr 172.21.15.3:0/750080389 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.321+0000 7ff068a00700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cde70 0x7ff06415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff069400700 1 -- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cde70 msgr2=0x7ff06415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff069400700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cde70 0x7ff06415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff069400700 1 -- 172.21.15.3:0/750080389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff064164f20 msgr2=0x7ff06415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff069400700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff064164f20 0x7ff06415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff069400700 1 -- 172.21.15.3:0/750080389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580094b0 con 0x7ff0640cb7c0 2024-09-13T16:50:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff069e00700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff064164f20 0x7ff06415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff069400700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cb7c0 0x7ff06415c3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff058016000 tx=0x7ff058004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff061600700 1 -- 172.21.15.3:0/750080389 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058004590 con 0x7ff0640cb7c0 2024-09-13T16:50:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff061600700 1 -- 172.21.15.3:0/750080389 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0580036c0 con 0x7ff0640cb7c0 2024-09-13T16:50:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff061600700 1 -- 172.21.15.3:0/750080389 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058003a70 con 0x7ff0640cb7c0 2024-09-13T16:50:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06415d520 con 0x7ff0640cb7c0 2024-09-13T16:50:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06415c900 con 0x7ff0640cb7c0 2024-09-13T16:50:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.325+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0640a8780 con 0x7ff0640cb7c0 2024-09-13T16:50:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.329+0000 7ff061600700 1 -- 172.21.15.3:0/750080389 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff058003bd0 con 0x7ff0640cb7c0 2024-09-13T16:50:54.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.329+0000 7ff061600700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff044066730 0x7ff044068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.329+0000 7ff061600700 1 -- 172.21.15.3:0/750080389 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff058021030 con 0x7ff0640cb7c0 2024-09-13T16:50:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.329+0000 7ff068a00700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff044066730 0x7ff044068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.333+0000 7ff068a00700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff044066730 0x7ff044068bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff06406dfd0 tx=0x7ff050009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.337+0000 7ff061600700 1 -- 172.21.15.3:0/750080389 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff0580529d0 con 0x7ff0640cb7c0 2024-09-13T16:50:54.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.561+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff0641649e0 con 0x7ff0640cb7c0 2024-09-13T16:50:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.697+0000 7ff061600700 1 -- 172.21.15.3:0/750080389 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff058055ff0 con 0x7ff0640cb7c0 2024-09-13T16:50:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff044066730 msgr2=0x7ff044068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff044066730 0x7ff044068bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff06406dfd0 tx=0x7ff050009240 comp rx=0 tx=0).stop 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cb7c0 msgr2=0x7ff06415c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cb7c0 0x7ff06415c3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff058016000 tx=0x7ff058004770 comp rx=0 tx=0).stop 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 shutdown_connections 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff044066730 0x7ff044068bf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff064164f20 0x7ff06415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0640cde70 0x7ff06415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 --2- 172.21.15.3:0/750080389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0640cb7c0 0x7ff06415c3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.701+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 >> 172.21.15.3:0/750080389 conn(0x7ff0640af620 msgr2=0x7ff064071620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.705+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 shutdown_connections 2024-09-13T16:50:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.705+0000 7ff06a800700 1 -- 172.21.15.3:0/750080389 wait complete. 2024-09-13T16:50:54.730 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-13T16:50:54.730 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T16:50:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.913+0000 7fe11e400700 1 Processor -- start 2024-09-13T16:50:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.913+0000 7fe11e400700 1 -- start start 2024-09-13T16:50:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe11e400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1181442c0 0x7fe11806d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe11e400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1180b41e0 con 0x7fe11806dcb0 2024-09-13T16:50:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe11e400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11806a390 con 0x7fe1180cded0 2024-09-13T16:50:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe11e400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11806b1f0 con 0x7fe1181442c0 2024-09-13T16:50:54.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe117e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe11806dcb0 0x7fe1180cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39400/0 (socket says 172.21.15.3:39400) 2024-09-13T16:50:54.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe117e00700 1 -- 172.21.15.3:0/3423048573 learned_addr learned my addr 172.21.15.3:0/3423048573 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:54.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe11d000700 1 --2- 172.21.15.3:0/3423048573 >> v2:172.21.15.157:3300/0 conn(0x7fe1181442c0 0x7fe11806d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe117400700 1 -- 172.21.15.3:0/3423048573 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355447122 0 0) 0x7fe11806a390 con 0x7fe1180cded0 2024-09-13T16:50:54.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe117400700 1 -- 172.21.15.3:0/3423048573 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1000035d0 con 0x7fe1180cded0 2024-09-13T16:50:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe117400700 1 -- 172.21.15.3:0/3423048573 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913019515 0 0) 0x7fe1180b41e0 con 0x7fe11806dcb0 2024-09-13T16:50:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe117400700 1 -- 172.21.15.3:0/3423048573 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe11806a390 con 0x7fe11806dcb0 2024-09-13T16:50:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe11d000700 1 -- 172.21.15.3:0/3423048573 >> v1:172.21.15.157:6789/0 conn(0x7fe1180cded0 legacy=0x7fe11815b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe117400700 1 -- 172.21.15.3:0/3423048573 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 394535040 0 0) 0x7fe1000035d0 con 0x7fe1180cded0 2024-09-13T16:50:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe11d000700 1 -- 172.21.15.3:0/3423048573 >> v1:172.21.15.3:6789/0 conn(0x7fe11806dcb0 legacy=0x7fe1180cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.917+0000 7fe11d000700 1 -- 172.21.15.3:0/3423048573 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11806a5e0 con 0x7fe1181442c0 2024-09-13T16:50:54.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe11d000700 1 --2- 172.21.15.3:0/3423048573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 0x7fe11806d770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0097d0 tx=0x7fe10c009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aea946cb1c4c6e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:54.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe117400700 1 -- 172.21.15.3:0/3423048573 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104001670 con 0x7fe1181442c0 2024-09-13T16:50:54.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe117400700 1 -- 172.21.15.3:0/3423048573 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c00f510 con 0x7fe1181442c0 2024-09-13T16:50:54.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe117400700 1 -- 172.21.15.3:0/3423048573 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100019f0 con 0x7fe1181442c0 2024-09-13T16:50:54.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe11e400700 1 -- 172.21.15.3:0/3423048573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 msgr2=0x7fe11806d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe11e400700 1 --2- 172.21.15.3:0/3423048573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 0x7fe11806d770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0097d0 tx=0x7fe10c009ae0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe11e400700 1 -- 172.21.15.3:0/3423048573 shutdown_connections 2024-09-13T16:50:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe11e400700 1 --2- 172.21.15.3:0/3423048573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 0x7fe11806d770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe11e400700 1 -- 172.21.15.3:0/3423048573 >> 172.21.15.3:0/3423048573 conn(0x7fe1180af620 msgr2=0x7fe1180afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe11e400700 1 -- 172.21.15.3:0/3423048573 shutdown_connections 2024-09-13T16:50:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.921+0000 7fe11e400700 1 -- 172.21.15.3:0/3423048573 wait complete. 2024-09-13T16:50:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.925+0000 7fe11e400700 1 Processor -- start 2024-09-13T16:50:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.925+0000 7fe11e400700 1 -- start start 2024-09-13T16:50:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11e400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe11806dcb0 0x7fe11814c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11e400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1180cded0 0x7fe11814e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11e400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 0x7fe11814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11e400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11814c270 con 0x7fe1180cded0 2024-09-13T16:50:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11e400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11815a440 con 0x7fe1181442c0 2024-09-13T16:50:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11e400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11814b5f0 con 0x7fe11806dcb0 2024-09-13T16:50:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1180cded0 0x7fe11814e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1180cded0 0x7fe11814e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54336/0 (socket says 172.21.15.3:54336) 2024-09-13T16:50:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 -- 172.21.15.3:0/1794853959 learned_addr learned my addr 172.21.15.3:0/1794853959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11da00700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 0x7fe11814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11d000700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe11806dcb0 0x7fe11814c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 -- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe11806dcb0 msgr2=0x7fe11814c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe11806dcb0 0x7fe11814c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 -- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 msgr2=0x7fe11814eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 0x7fe11814eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 -- 172.21.15.3:0/1794853959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0094b0 con 0x7fe1180cded0 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11d000700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe11806dcb0 0x7fe11814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe117e00700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1180cded0 0x7fe11814e570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe1040011e0 tx=0x7fe10400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe114c00700 1 -- 172.21.15.3:0/1794853959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104004ca0 con 0x7fe1180cded0 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe114c00700 1 -- 172.21.15.3:0/1794853959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe104004e00 con 0x7fe1180cded0 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe114c00700 1 -- 172.21.15.3:0/1794853959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104007310 con 0x7fe1180cded0 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.929+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11814b8d0 con 0x7fe1180cded0 2024-09-13T16:50:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.933+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11814be90 con 0x7fe1180cded0 2024-09-13T16:50:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.933+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11814c3b0 con 0x7fe1180cded0 2024-09-13T16:50:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.937+0000 7fe114c00700 1 -- 172.21.15.3:0/1794853959 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe104007470 con 0x7fe1180cded0 2024-09-13T16:50:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.937+0000 7fe114c00700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe0f8066730 0x7fe0f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.937+0000 7fe114c00700 1 -- 172.21.15.3:0/1794853959 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe1040842e0 con 0x7fe1180cded0 2024-09-13T16:50:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.937+0000 7fe11d000700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe0f8066730 0x7fe0f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.937+0000 7fe11d000700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe0f8066730 0x7fe0f8068bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe10c000c00 tx=0x7fe10c008000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:54.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:54.941+0000 7fe114c00700 1 -- 172.21.15.3:0/1794853959 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe11814c3b0 con 0x7fe1180cded0 2024-09-13T16:50:55.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.169+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe11814c3b0 con 0x7fe1180cded0 2024-09-13T16:50:55.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.709+0000 7fe114c00700 1 -- 172.21.15.3:0/1794853959 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe11814c3b0 con 0x7fe1180cded0 2024-09-13T16:50:55.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe0f8066730 msgr2=0x7fe0f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe0f8066730 0x7fe0f8068bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe10c000c00 tx=0x7fe10c008000 comp rx=0 tx=0).stop 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1180cded0 msgr2=0x7fe11814e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1180cded0 0x7fe11814e570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe1040011e0 tx=0x7fe10400c9b0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 shutdown_connections 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe0f8066730 0x7fe0f8068bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1181442c0 0x7fe11814eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1180cded0 0x7fe11814e570 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 --2- 172.21.15.3:0/1794853959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe11806dcb0 0x7fe11814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 >> 172.21.15.3:0/1794853959 conn(0x7fe1180af620 msgr2=0x7fe11815cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 shutdown_connections 2024-09-13T16:50:55.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.717+0000 7fe11e400700 1 -- 172.21.15.3:0/1794853959 wait complete. 2024-09-13T16:50:55.745 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T16:50:55.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:55.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.921+0000 7fea01800700 1 Processor -- start 2024-09-13T16:50:55.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.921+0000 7fea01800700 1 -- start start 2024-09-13T16:50:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.921+0000 7fea01800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc1443b0 0x7fe9fc0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.921+0000 7fea01800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9fc144790 0x7fe9fc06c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fea01800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc06cf70 con 0x7fe9fc06cb20 2024-09-13T16:50:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fea01800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9fc0ce8e0 con 0x7fe9fc144790 2024-09-13T16:50:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fea01800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0ccd30 con 0x7fe9fc1443b0 2024-09-13T16:50:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9f3400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9fc144790 0x7fe9fc06c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fbe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc1443b0 0x7fe9fc0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fbe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc1443b0 0x7fe9fc0b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52424/0 (socket says 172.21.15.3:52424) 2024-09-13T16:50:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fbe00700 1 -- 172.21.15.3:0/999206560 learned_addr learned my addr 172.21.15.3:0/999206560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fbe00700 1 -- 172.21.15.3:0/999206560 >> v2:172.21.15.3:3300/0 conn(0x7fe9fc144790 msgr2=0x7fe9fc06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fbe00700 1 --2- 172.21.15.3:0/999206560 >> v2:172.21.15.3:3300/0 conn(0x7fe9fc144790 0x7fe9fc06c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fbe00700 1 -- 172.21.15.3:0/999206560 >> v1:172.21.15.157:6789/0 conn(0x7fe9fc06cb20 legacy=0x7fe9fc06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fbe00700 1 -- 172.21.15.3:0/999206560 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0cd010 con 0x7fe9fc1443b0 2024-09-13T16:50:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9f3400700 1 --2- 172.21.15.3:0/999206560 >> v2:172.21.15.3:3300/0 conn(0x7fe9fc144790 0x7fe9fc06c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fbe00700 1 --2- 172.21.15.3:0/999206560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc1443b0 0x7fe9fc0b39f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009910 tx=0x7fe9ec009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9bf80b2903a4d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fb400700 1 -- 172.21.15.3:0/999206560 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec011830 con 0x7fe9fc1443b0 2024-09-13T16:50:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fb400700 1 -- 172.21.15.3:0/999206560 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec011990 con 0x7fe9fc1443b0 2024-09-13T16:50:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.925+0000 7fe9fb400700 1 -- 172.21.15.3:0/999206560 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec011d40 con 0x7fe9fc1443b0 2024-09-13T16:50:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 -- 172.21.15.3:0/999206560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc1443b0 msgr2=0x7fe9fc0b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:55.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 --2- 172.21.15.3:0/999206560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc1443b0 0x7fe9fc0b39f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009910 tx=0x7fe9ec009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:55.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 -- 172.21.15.3:0/999206560 shutdown_connections 2024-09-13T16:50:55.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 --2- 172.21.15.3:0/999206560 >> v2:172.21.15.3:3300/0 conn(0x7fe9fc144790 0x7fe9fc06c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 --2- 172.21.15.3:0/999206560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc1443b0 0x7fe9fc0b39f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 -- 172.21.15.3:0/999206560 >> 172.21.15.3:0/999206560 conn(0x7fe9fc0b0cf0 msgr2=0x7fe9fc0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 -- 172.21.15.3:0/999206560 shutdown_connections 2024-09-13T16:50:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 -- 172.21.15.3:0/999206560 wait complete. 2024-09-13T16:50:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 Processor -- start 2024-09-13T16:50:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.929+0000 7fea01800700 1 -- start start 2024-09-13T16:50:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea01800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc06cb20 0x7fe9fc13a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea01800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9fc1443b0 0x7fe9fc139a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea01800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9fc139fe0 0x7fe9fc0ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea01800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0adb80 con 0x7fe9fc1443b0 2024-09-13T16:50:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea01800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0acf70 con 0x7fe9fc139fe0 2024-09-13T16:50:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea01800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0ad250 con 0x7fe9fc06cb20 2024-09-13T16:50:55.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9fc1443b0 0x7fe9fc139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:55.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9fc1443b0 0x7fe9fc139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54348/0 (socket says 172.21.15.3:54348) 2024-09-13T16:50:55.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 -- 172.21.15.3:0/947845247 learned_addr learned my addr 172.21.15.3:0/947845247 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:55.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9fbe00700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc06cb20 0x7fe9fc13a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:55.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea00e00700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9fc139fe0 0x7fe9fc0ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:55.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 -- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc06cb20 msgr2=0x7fe9fc13a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:55.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc06cb20 0x7fe9fc13a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 -- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9fc139fe0 msgr2=0x7fe9fc0ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:55.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9fc139fe0 0x7fe9fc0ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:55.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 -- 172.21.15.3:0/947845247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e40094b0 con 0x7fe9fc1443b0 2024-09-13T16:50:55.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f3400700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9fc1443b0 0x7fe9fc139a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400ebe0 tx=0x7fe9e400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:55.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f8c00700 1 -- 172.21.15.3:0/947845247 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400c850 con 0x7fe9fc1443b0 2024-09-13T16:50:55.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f8c00700 1 -- 172.21.15.3:0/947845247 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e400c9b0 con 0x7fe9fc1443b0 2024-09-13T16:50:55.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec009570 con 0x7fe9fc1443b0 2024-09-13T16:50:55.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fe9f8c00700 1 -- 172.21.15.3:0/947845247 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400cd40 con 0x7fe9fc1443b0 2024-09-13T16:50:55.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.933+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc0ad810 con 0x7fe9fc1443b0 2024-09-13T16:50:55.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.937+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc0adcf0 con 0x7fe9fc1443b0 2024-09-13T16:50:55.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.937+0000 7fe9f8c00700 1 -- 172.21.15.3:0/947845247 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe9e4013470 con 0x7fe9fc1443b0 2024-09-13T16:50:55.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.937+0000 7fe9f8c00700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe9d8066760 0x7fe9d8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:55.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.937+0000 7fe9f8c00700 1 -- 172.21.15.3:0/947845247 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe9e4084520 con 0x7fe9fc1443b0 2024-09-13T16:50:55.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.937+0000 7fe9fbe00700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe9d8066760 0x7fe9d8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.941+0000 7fe9fbe00700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe9d8066760 0x7fe9d8068c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000c00 tx=0x7fe9ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:55.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:55.945+0000 7fe9f8c00700 1 -- 172.21.15.3:0/947845247 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe9e4051500 con 0x7fe9fc1443b0 2024-09-13T16:50:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.161+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9fc06c5e0 con 0x7fe9fc1443b0 2024-09-13T16:50:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.161+0000 7fe9f8c00700 1 -- 172.21.15.3:0/947845247 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4727 (secure 0 0 0) 0x7fe9e400cea0 con 0x7fe9fc1443b0 2024-09-13T16:50:56.167 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:56.167 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:55.700655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:50:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe9d8066760 msgr2=0x7fe9d8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe9d8066760 0x7fe9d8068c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000c00 tx=0x7fe9ec008030 comp rx=0 tx=0).stop 2024-09-13T16:50:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9fc1443b0 msgr2=0x7fe9fc139a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9fc1443b0 0x7fe9fc139a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400ebe0 tx=0x7fe9e400eef0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 shutdown_connections 2024-09-13T16:50:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe9d8066760 0x7fe9d8068c20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9fc139fe0 0x7fe9fc0ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9fc1443b0 0x7fe9fc139a00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 --2- 172.21.15.3:0/947845247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9fc06cb20 0x7fe9fc13a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.165+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 >> 172.21.15.3:0/947845247 conn(0x7fe9fc0b0cf0 msgr2=0x7fe9fc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.169+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 shutdown_connections 2024-09-13T16:50:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.169+0000 7fea01800700 1 -- 172.21.15.3:0/947845247 wait complete. 2024-09-13T16:50:56.174 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-13T16:50:56.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:56.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.361+0000 7f0909200700 1 Processor -- start 2024-09-13T16:50:56.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.361+0000 7f0909200700 1 -- start start 2024-09-13T16:50:56.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0909200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0904071240 0x7f090406cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0909200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040716f0 con 0x7f0904070db0 2024-09-13T16:50:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0909200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040b3450 con 0x7f090415b660 2024-09-13T16:50:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0909200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040b40d0 con 0x7f0904071240 2024-09-13T16:50:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f08f9400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0904070db0 0x7f090406c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39406/0 (socket says 172.21.15.3:39406) 2024-09-13T16:50:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f08f9400700 1 -- 172.21.15.3:0/4117427667 learned_addr learned my addr 172.21.15.3:0/4117427667 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0902800700 1 --2- 172.21.15.3:0/4117427667 >> v2:172.21.15.157:3300/0 conn(0x7f0904071240 0x7f090406cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0902800700 1 -- 172.21.15.3:0/4117427667 >> v1:172.21.15.157:6789/0 conn(0x7f090415b660 legacy=0x7f09040b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0901400700 1 -- 172.21.15.3:0/4117427667 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203909187 0 0) 0x7f09040716f0 con 0x7f0904070db0 2024-09-13T16:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0902800700 1 -- 172.21.15.3:0/4117427667 >> v1:172.21.15.3:6789/0 conn(0x7f0904070db0 legacy=0x7f090406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0902800700 1 -- 172.21.15.3:0/4117427667 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040b3730 con 0x7f0904071240 2024-09-13T16:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0902800700 1 --2- 172.21.15.3:0/4117427667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 0x7f090406cc50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009850 tx=0x7f08fc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4547115f674dde server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0901400700 1 -- 172.21.15.3:0/4117427667 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc011830 con 0x7f0904071240 2024-09-13T16:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0901400700 1 -- 172.21.15.3:0/4117427667 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc011990 con 0x7f0904071240 2024-09-13T16:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.365+0000 7f0901400700 1 -- 172.21.15.3:0/4117427667 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc011d20 con 0x7f0904071240 2024-09-13T16:50:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 -- 172.21.15.3:0/4117427667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 msgr2=0x7f090406cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 --2- 172.21.15.3:0/4117427667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 0x7f090406cc50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009850 tx=0x7f08fc009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 -- 172.21.15.3:0/4117427667 shutdown_connections 2024-09-13T16:50:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 --2- 172.21.15.3:0/4117427667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 0x7f090406cc50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 -- 172.21.15.3:0/4117427667 >> 172.21.15.3:0/4117427667 conn(0x7f090406a390 msgr2=0x7f090406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 -- 172.21.15.3:0/4117427667 shutdown_connections 2024-09-13T16:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 -- 172.21.15.3:0/4117427667 wait complete. 2024-09-13T16:50:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 Processor -- start 2024-09-13T16:50:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.369+0000 7f0909200700 1 -- start start 2024-09-13T16:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0909200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0904070db0 0x7f0904156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0909200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 0x7f090415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0909200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f090415b660 0x7f090415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0909200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040ae150 con 0x7f090415b660 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0909200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090415bd80 con 0x7f0904071240 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0909200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090415c060 con 0x7f0904070db0 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f090415b660 0x7f090415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f090415b660 0x7f090415e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54358/0 (socket says 172.21.15.3:54358) 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 -- 172.21.15.3:0/3589439489 learned_addr learned my addr 172.21.15.3:0/3589439489 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f08f9400700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 0x7f090415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0901e00700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0904070db0 0x7f0904156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 -- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0904070db0 msgr2=0x7f0904156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0904070db0 0x7f0904156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 -- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 msgr2=0x7f090415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 0x7f090415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 -- 172.21.15.3:0/3589439489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc0094b0 con 0x7f090415b660 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0901e00700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0904070db0 0x7f0904156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f0902800700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f090415b660 0x7f090415e5b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0090b0 tx=0x7f08fc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f08faa00700 1 -- 172.21.15.3:0/3589439489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00ba70 con 0x7f090415b660 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.373+0000 7f08faa00700 1 -- 172.21.15.3:0/3589439489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc01f070 con 0x7f090415b660 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.377+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090415c340 con 0x7f090415b660 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.377+0000 7f08faa00700 1 -- 172.21.15.3:0/3589439489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00be30 con 0x7f090415b660 2024-09-13T16:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.377+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090415f0a0 con 0x7f090415b660 2024-09-13T16:50:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.377+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090415c860 con 0x7f090415b660 2024-09-13T16:50:56.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.377+0000 7f08faa00700 1 -- 172.21.15.3:0/3589439489 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08fc01a430 con 0x7f090415b660 2024-09-13T16:50:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.381+0000 7f08faa00700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08e0066730 0x7f08e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.381+0000 7f08faa00700 1 -- 172.21.15.3:0/3589439489 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f08fc021070 con 0x7f090415b660 2024-09-13T16:50:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.381+0000 7f0901e00700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08e0066730 0x7f08e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.381+0000 7f0901e00700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08e0066730 0x7f08e0068bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08f40012e0 tx=0x7f08f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:56.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.385+0000 7f08faa00700 1 -- 172.21.15.3:0/3589439489 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f090415c860 con 0x7f090415b660 2024-09-13T16:50:56.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.577+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f090415c860 con 0x7f090415b660 2024-09-13T16:50:56.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.581+0000 7f08faa00700 1 -- 172.21.15.3:0/3589439489 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7f090415c860 con 0x7f090415b660 2024-09-13T16:50:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:56.586 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08e0066730 msgr2=0x7f08e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08e0066730 0x7f08e0068bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08f40012e0 tx=0x7f08f4009040 comp rx=0 tx=0).stop 2024-09-13T16:50:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f090415b660 msgr2=0x7f090415e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f090415b660 0x7f090415e5b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0090b0 tx=0x7f08fc004770 comp rx=0 tx=0).stop 2024-09-13T16:50:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 shutdown_connections 2024-09-13T16:50:56.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08e0066730 0x7f08e0068bf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f090415b660 0x7f090415e5b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904071240 0x7f090415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 --2- 172.21.15.3:0/3589439489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0904070db0 0x7f0904156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 >> 172.21.15.3:0/3589439489 conn(0x7f090406a390 msgr2=0x7f09040cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:56.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 shutdown_connections 2024-09-13T16:50:56.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.585+0000 7f0909200700 1 -- 172.21.15.3:0/3589439489 wait complete. 2024-09-13T16:50:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T16:50:56.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:50:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.789+0000 7f6f93e00700 1 Processor -- start 2024-09-13T16:50:56.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.789+0000 7f6f93e00700 1 -- start start 2024-09-13T16:50:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f93e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f9415cdd0 0x7f6f940d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f93e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415d1b0 0x7f6f940b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f93e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f940b8d40 con 0x7f6f940d25b0 2024-09-13T16:50:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f93e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f940d1510 con 0x7f6f9415cdd0 2024-09-13T16:50:56.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f93e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940d17f0 con 0x7f6f9415d1b0 2024-09-13T16:50:56.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f92a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f9415cdd0 0x7f6f940d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f92a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f9415cdd0 0x7f6f940d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54372/0 (socket says 172.21.15.3:54372) 2024-09-13T16:50:56.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f92a00700 1 -- 172.21.15.3:0/2157021446 learned_addr learned my addr 172.21.15.3:0/2157021446 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:56.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f92000700 1 --2- 172.21.15.3:0/2157021446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415d1b0 0x7f6f940b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f92a00700 1 -- 172.21.15.3:0/2157021446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415d1b0 msgr2=0x7f6f940b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:56.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f92a00700 1 --2- 172.21.15.3:0/2157021446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415d1b0 0x7f6f940b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f92a00700 1 -- 172.21.15.3:0/2157021446 >> v1:172.21.15.157:6789/0 conn(0x7f6f940d25b0 legacy=0x7f6f940cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.793+0000 7f6f92a00700 1 -- 172.21.15.3:0/2157021446 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f940d1ad0 con 0x7f6f9415cdd0 2024-09-13T16:50:56.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f92a00700 1 --2- 172.21.15.3:0/2157021446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9415cdd0 0x7f6f940d0e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009910 tx=0x7f6f84009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd8952674ef377d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:56.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f91600700 1 -- 172.21.15.3:0/2157021446 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84011830 con 0x7f6f9415cdd0 2024-09-13T16:50:56.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f91600700 1 -- 172.21.15.3:0/2157021446 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f84011990 con 0x7f6f9415cdd0 2024-09-13T16:50:56.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f91600700 1 -- 172.21.15.3:0/2157021446 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84011d40 con 0x7f6f9415cdd0 2024-09-13T16:50:56.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f93e00700 1 -- 172.21.15.3:0/2157021446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9415cdd0 msgr2=0x7f6f940d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2157021446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9415cdd0 0x7f6f940d0e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009910 tx=0x7f6f84009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:56.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f93e00700 1 -- 172.21.15.3:0/2157021446 shutdown_connections 2024-09-13T16:50:56.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2157021446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415d1b0 0x7f6f940b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2157021446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9415cdd0 0x7f6f940d0e10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f93e00700 1 -- 172.21.15.3:0/2157021446 >> 172.21.15.3:0/2157021446 conn(0x7f6f940d3190 msgr2=0x7f6f940d35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:56.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f93e00700 1 -- 172.21.15.3:0/2157021446 shutdown_connections 2024-09-13T16:50:56.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.797+0000 7f6f93e00700 1 -- 172.21.15.3:0/2157021446 wait complete. 2024-09-13T16:50:56.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93e00700 1 Processor -- start 2024-09-13T16:50:56.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93e00700 1 -- start start 2024-09-13T16:50:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f940d25b0 0x7f6f94150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415cdd0 0x7f6f940c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9415d1b0 0x7f6f940cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940ca450 con 0x7f6f940d25b0 2024-09-13T16:50:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940c9e40 con 0x7f6f9415d1b0 2024-09-13T16:50:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940c8510 con 0x7f6f9415cdd0 2024-09-13T16:50:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f940d25b0 0x7f6f94150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f940d25b0 0x7f6f94150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54382/0 (socket says 172.21.15.3:54382) 2024-09-13T16:50:56.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 -- 172.21.15.3:0/2427458070 learned_addr learned my addr 172.21.15.3:0/2427458070 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 -- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415cdd0 msgr2=0x7f6f940c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415cdd0 0x7f6f940c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 -- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9415d1b0 msgr2=0x7f6f940cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93400700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9415d1b0 0x7f6f940cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9415d1b0 0x7f6f940cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 -- 172.21.15.3:0/2427458070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f84009570 con 0x7f6f940d25b0 2024-09-13T16:50:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f93400700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9415d1b0 0x7f6f940cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.801+0000 7f6f92a00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f940d25b0 0x7f6f94150250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000c00 tx=0x7f6f84004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.805+0000 7f6f8aa00700 1 -- 172.21.15.3:0/2427458070 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84004610 con 0x7f6f940d25b0 2024-09-13T16:50:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.805+0000 7f6f8aa00700 1 -- 172.21.15.3:0/2427458070 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f840036c0 con 0x7f6f940d25b0 2024-09-13T16:50:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.805+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f940d2be0 con 0x7f6f940d25b0 2024-09-13T16:50:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.805+0000 7f6f8aa00700 1 -- 172.21.15.3:0/2427458070 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84003a70 con 0x7f6f940d25b0 2024-09-13T16:50:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.805+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f940ca0e0 con 0x7f6f940d25b0 2024-09-13T16:50:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.805+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f940b8800 con 0x7f6f940d25b0 2024-09-13T16:50:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.805+0000 7f6f8aa00700 1 -- 172.21.15.3:0/2427458070 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f84003bd0 con 0x7f6f940d25b0 2024-09-13T16:50:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.809+0000 7f6f8aa00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6f74066730 0x7f6f74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.809+0000 7f6f8aa00700 1 -- 172.21.15.3:0/2427458070 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6f84021030 con 0x7f6f940d25b0 2024-09-13T16:50:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.809+0000 7f6f92000700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6f74066730 0x7f6f74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:56.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.809+0000 7f6f92000700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6f74066730 0x7f6f74068bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f80006b40 tx=0x7f6f80006ad0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:56.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:56.813+0000 7f6f8aa00700 1 -- 172.21.15.3:0/2427458070 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6f84052c50 con 0x7f6f940d25b0 2024-09-13T16:50:57.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.009+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f940c8a30 con 0x7f6f74066730 2024-09-13T16:50:57.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.033+0000 7f6f8aa00700 1 -- 172.21.15.3:0/2427458070 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345056 (secure 0 0 0) 0x7f6f800096d0 con 0x7f6f74066730 2024-09-13T16:50:57.042 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6f74066730 msgr2=0x7f6f74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6f74066730 0x7f6f74068bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f80006b40 tx=0x7f6f80006ad0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f940d25b0 msgr2=0x7f6f94150250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f940d25b0 0x7f6f94150250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000c00 tx=0x7f6f84004770 comp rx=0 tx=0).stop 2024-09-13T16:50:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 shutdown_connections 2024-09-13T16:50:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6f74066730 0x7f6f74068bf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9415d1b0 0x7f6f940cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9415cdd0 0x7f6f940c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 --2- 172.21.15.3:0/2427458070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f940d25b0 0x7f6f94150250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.037+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 >> 172.21.15.3:0/2427458070 conn(0x7f6f940d3190 msgr2=0x7f6f940cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:57.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.041+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 shutdown_connections 2024-09-13T16:50:57.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.041+0000 7f6f93e00700 1 -- 172.21.15.3:0/2427458070 wait complete. 2024-09-13T16:50:57.048 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:50:57.067 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-13T16:50:55.200694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227664,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754747056,"statfs":{"total":773094113280,"available":772860985344,"internally_reserved":0,"allocated":3383296,"data_stored":2044682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2231,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005046"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463496+0000","last_change":"2024-09-13T16:50:30.106124+0000","last_active":"2024-09-13T16:50:35.463496+0000","last_peered":"2024-09-13T16:50:35.463496+0000","last_clean":"2024-09-13T16:50:35.463496+0000","last_became_active":"2024-09-13T16:50:30.105874+0000","last_became_peered":"2024-09-13T16:50:30.105874+0000","last_unstale":"2024-09-13T16:50:35.463496+0000","last_undegraded":"2024-09-13T16:50:35.463496+0000","last_fullsized":"2024-09-13T16:50:35.463496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:38.882189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456272+0000","last_change":"2024-09-13T16:50:30.598817+0000","last_active":"2024-09-13T16:50:35.456272+0000","last_peered":"2024-09-13T16:50:35.456272+0000","last_clean":"2024-09-13T16:50:35.456272+0000","last_became_active":"2024-09-13T16:50:30.598483+0000","last_became_peered":"2024-09-13T16:50:30.598483+0000","last_unstale":"2024-09-13T16:50:35.456272+0000","last_undegraded":"2024-09-13T16:50:35.456272+0000","last_fullsized":"2024-09-13T16:50:35.456272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.511304+0000","last_change":"2024-09-13T16:50:35.507978+0000","last_active":"2024-09-13T16:50:35.511304+0000","last_peered":"2024-09-13T16:50:35.511304+0000","last_clean":"2024-09-13T16:50:35.511304+0000","last_became_active":"2024-09-13T16:50:35.506655+0000","last_became_peered":"2024-09-13T16:50:35.506655+0000","last_unstale":"2024-09-13T16:50:35.511304+0000","last_undegraded":"2024-09-13T16:50:35.511304+0000","last_fullsized":"2024-09-13T16:50:35.511304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:29.062934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456822+0000","last_change":"2024-09-13T16:50:30.101330+0000","last_active":"2024-09-13T16:50:35.456822+0000","last_peered":"2024-09-13T16:50:35.456822+0000","last_clean":"2024-09-13T16:50:35.456822+0000","last_became_active":"2024-09-13T16:50:30.101038+0000","last_became_peered":"2024-09-13T16:50:30.101038+0000","last_unstale":"2024-09-13T16:50:35.456822+0000","last_undegraded":"2024-09-13T16:50:35.456822+0000","last_fullsized":"2024-09-13T16:50:35.456822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463263+0000","last_change":"2024-09-13T16:50:30.609337+0000","last_active":"2024-09-13T16:50:35.463263+0000","last_peered":"2024-09-13T16:50:35.463263+0000","last_clean":"2024-09-13T16:50:35.463263+0000","last_became_active":"2024-09-13T16:50:30.609098+0000","last_became_peered":"2024-09-13T16:50:30.609098+0000","last_unstale":"2024-09-13T16:50:35.463263+0000","last_undegraded":"2024-09-13T16:50:35.463263+0000","last_fullsized":"2024-09-13T16:50:35.463263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498079+0000","last_change":"2024-09-13T16:50:34.633734+0000","last_active":"2024-09-13T16:50:35.498079+0000","last_peered":"2024-09-13T16:50:35.498079+0000","last_clean":"2024-09-13T16:50:35.498079+0000","last_became_active":"2024-09-13T16:50:34.632931+0000","last_became_peered":"2024-09-13T16:50:34.632931+0000","last_unstale":"2024-09-13T16:50:35.498079+0000","last_undegraded":"2024-09-13T16:50:35.498079+0000","last_fullsized":"2024-09-13T16:50:35.498079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:37.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500017+0000","last_change":"2024-09-13T16:50:30.108543+0000","last_active":"2024-09-13T16:50:35.500017+0000","last_peered":"2024-09-13T16:50:35.500017+0000","last_clean":"2024-09-13T16:50:35.500017+0000","last_became_active":"2024-09-13T16:50:30.108282+0000","last_became_peered":"2024-09-13T16:50:30.108282+0000","last_unstale":"2024-09-13T16:50:35.500017+0000","last_undegraded":"2024-09-13T16:50:35.500017+0000","last_fullsized":"2024-09-13T16:50:35.500017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092867+0000","last_change":"2024-09-13T16:50:30.594489+0000","last_active":"2024-09-13T16:50:35.092867+0000","last_peered":"2024-09-13T16:50:35.092867+0000","last_clean":"2024-09-13T16:50:35.092867+0000","last_became_active":"2024-09-13T16:50:30.593342+0000","last_became_peered":"2024-09-13T16:50:30.593342+0000","last_unstale":"2024-09-13T16:50:35.092867+0000","last_undegraded":"2024-09-13T16:50:35.092867+0000","last_fullsized":"2024-09-13T16:50:35.092867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500758+0000","last_change":"2024-09-13T16:50:34.633900+0000","last_active":"2024-09-13T16:50:35.500758+0000","last_peered":"2024-09-13T16:50:35.500758+0000","last_clean":"2024-09-13T16:50:35.500758+0000","last_became_active":"2024-09-13T16:50:34.633428+0000","last_became_peered":"2024-09-13T16:50:34.633428+0000","last_unstale":"2024-09-13T16:50:35.500758+0000","last_undegraded":"2024-09-13T16:50:35.500758+0000","last_fullsized":"2024-09-13T16:50:35.500758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:01.459147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499214+0000","last_change":"2024-09-13T16:50:30.119534+0000","last_active":"2024-09-13T16:50:35.499214+0000","last_peered":"2024-09-13T16:50:35.499214+0000","last_clean":"2024-09-13T16:50:35.499214+0000","last_became_active":"2024-09-13T16:50:30.119177+0000","last_became_peered":"2024-09-13T16:50:30.119177+0000","last_unstale":"2024-09-13T16:50:35.499214+0000","last_undegraded":"2024-09-13T16:50:35.499214+0000","last_fullsized":"2024-09-13T16:50:35.499214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092738+0000","last_change":"2024-09-13T16:50:30.604539+0000","last_active":"2024-09-13T16:50:35.092738+0000","last_peered":"2024-09-13T16:50:35.092738+0000","last_clean":"2024-09-13T16:50:35.092738+0000","last_became_active":"2024-09-13T16:50:30.604192+0000","last_became_peered":"2024-09-13T16:50:30.604192+0000","last_unstale":"2024-09-13T16:50:35.092738+0000","last_undegraded":"2024-09-13T16:50:35.092738+0000","last_fullsized":"2024-09-13T16:50:35.092738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500774+0000","last_change":"2024-09-13T16:50:34.632525+0000","last_active":"2024-09-13T16:50:35.500774+0000","last_peered":"2024-09-13T16:50:35.500774+0000","last_clean":"2024-09-13T16:50:35.500774+0000","last_became_active":"2024-09-13T16:50:34.632114+0000","last_became_peered":"2024-09-13T16:50:34.632114+0000","last_unstale":"2024-09-13T16:50:35.500774+0000","last_undegraded":"2024-09-13T16:50:35.500774+0000","last_fullsized":"2024-09-13T16:50:35.500774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091628+0000","last_change":"2024-09-13T16:50:30.611124+0000","last_active":"2024-09-13T16:50:35.091628+0000","last_peered":"2024-09-13T16:50:35.091628+0000","last_clean":"2024-09-13T16:50:35.091628+0000","last_became_active":"2024-09-13T16:50:30.607850+0000","last_became_peered":"2024-09-13T16:50:30.607850+0000","last_unstale":"2024-09-13T16:50:35.091628+0000","last_undegraded":"2024-09-13T16:50:35.091628+0000","last_fullsized":"2024-09-13T16:50:35.091628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499169+0000","last_change":"2024-09-13T16:50:30.106841+0000","last_active":"2024-09-13T16:50:35.499169+0000","last_peered":"2024-09-13T16:50:35.499169+0000","last_clean":"2024-09-13T16:50:35.499169+0000","last_became_active":"2024-09-13T16:50:30.106581+0000","last_became_peered":"2024-09-13T16:50:30.106581+0000","last_unstale":"2024-09-13T16:50:35.499169+0000","last_undegraded":"2024-09-13T16:50:35.499169+0000","last_fullsized":"2024-09-13T16:50:35.499169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501399+0000","last_change":"2024-09-13T16:50:34.633477+0000","last_active":"2024-09-13T16:50:35.501399+0000","last_peered":"2024-09-13T16:50:35.501399+0000","last_clean":"2024-09-13T16:50:35.501399+0000","last_became_active":"2024-09-13T16:50:34.633072+0000","last_became_peered":"2024-09-13T16:50:34.633072+0000","last_unstale":"2024-09-13T16:50:35.501399+0000","last_undegraded":"2024-09-13T16:50:35.501399+0000","last_fullsized":"2024-09-13T16:50:35.501399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:14.360209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456436+0000","last_change":"2024-09-13T16:50:30.611139+0000","last_active":"2024-09-13T16:50:35.456436+0000","last_peered":"2024-09-13T16:50:35.456436+0000","last_clean":"2024-09-13T16:50:35.456436+0000","last_became_active":"2024-09-13T16:50:30.610455+0000","last_became_peered":"2024-09-13T16:50:30.610455+0000","last_unstale":"2024-09-13T16:50:35.456436+0000","last_undegraded":"2024-09-13T16:50:35.456436+0000","last_fullsized":"2024-09-13T16:50:35.456436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402349+0000","last_change":"2024-09-13T16:50:30.113719+0000","last_active":"2024-09-13T16:50:35.402349+0000","last_peered":"2024-09-13T16:50:35.402349+0000","last_clean":"2024-09-13T16:50:35.402349+0000","last_became_active":"2024-09-13T16:50:30.111437+0000","last_became_peered":"2024-09-13T16:50:30.111437+0000","last_unstale":"2024-09-13T16:50:35.402349+0000","last_undegraded":"2024-09-13T16:50:35.402349+0000","last_fullsized":"2024-09-13T16:50:35.402349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501838+0000","last_change":"2024-09-13T16:50:34.638895+0000","last_active":"2024-09-13T16:50:35.501838+0000","last_peered":"2024-09-13T16:50:35.501838+0000","last_clean":"2024-09-13T16:50:35.501838+0000","last_became_active":"2024-09-13T16:50:34.638431+0000","last_became_peered":"2024-09-13T16:50:34.638431+0000","last_unstale":"2024-09-13T16:50:35.501838+0000","last_undegraded":"2024-09-13T16:50:35.501838+0000","last_fullsized":"2024-09-13T16:50:35.501838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:51.705201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.457086+0000","last_change":"2024-09-13T16:50:30.599950+0000","last_active":"2024-09-13T16:50:35.457086+0000","last_peered":"2024-09-13T16:50:35.457086+0000","last_clean":"2024-09-13T16:50:35.457086+0000","last_became_active":"2024-09-13T16:50:30.599401+0000","last_became_peered":"2024-09-13T16:50:30.599401+0000","last_unstale":"2024-09-13T16:50:35.457086+0000","last_undegraded":"2024-09-13T16:50:35.457086+0000","last_fullsized":"2024-09-13T16:50:35.457086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401297+0000","last_change":"2024-09-13T16:50:30.110698+0000","last_active":"2024-09-13T16:50:35.401297+0000","last_peered":"2024-09-13T16:50:35.401297+0000","last_clean":"2024-09-13T16:50:35.401297+0000","last_became_active":"2024-09-13T16:50:30.110343+0000","last_became_peered":"2024-09-13T16:50:30.110343+0000","last_unstale":"2024-09-13T16:50:35.401297+0000","last_undegraded":"2024-09-13T16:50:35.401297+0000","last_fullsized":"2024-09-13T16:50:35.401297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.512080+0000","last_change":"2024-09-13T16:50:35.510605+0000","last_active":"2024-09-13T16:50:35.512080+0000","last_peered":"2024-09-13T16:50:35.512080+0000","last_clean":"2024-09-13T16:50:35.512080+0000","last_became_active":"2024-09-13T16:50:35.510230+0000","last_became_peered":"2024-09-13T16:50:35.510230+0000","last_unstale":"2024-09-13T16:50:35.512080+0000","last_undegraded":"2024-09-13T16:50:35.512080+0000","last_fullsized":"2024-09-13T16:50:35.512080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500916+0000","last_change":"2024-09-13T16:50:30.608538+0000","last_active":"2024-09-13T16:50:35.500916+0000","last_peered":"2024-09-13T16:50:35.500916+0000","last_clean":"2024-09-13T16:50:35.500916+0000","last_became_active":"2024-09-13T16:50:30.608195+0000","last_became_peered":"2024-09-13T16:50:30.608195+0000","last_unstale":"2024-09-13T16:50:35.500916+0000","last_undegraded":"2024-09-13T16:50:35.500916+0000","last_fullsized":"2024-09-13T16:50:35.500916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:48.431406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.497470+0000","last_change":"2024-09-13T16:50:30.110135+0000","last_active":"2024-09-13T16:50:35.497470+0000","last_peered":"2024-09-13T16:50:35.497470+0000","last_clean":"2024-09-13T16:50:35.497470+0000","last_became_active":"2024-09-13T16:50:30.109520+0000","last_became_peered":"2024-09-13T16:50:30.109520+0000","last_unstale":"2024-09-13T16:50:35.497470+0000","last_undegraded":"2024-09-13T16:50:35.497470+0000","last_fullsized":"2024-09-13T16:50:35.497470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501389+0000","last_change":"2024-09-13T16:50:34.635782+0000","last_active":"2024-09-13T16:50:35.501389+0000","last_peered":"2024-09-13T16:50:35.501389+0000","last_clean":"2024-09-13T16:50:35.501389+0000","last_became_active":"2024-09-13T16:50:34.635349+0000","last_became_peered":"2024-09-13T16:50:34.635349+0000","last_unstale":"2024-09-13T16:50:35.501389+0000","last_undegraded":"2024-09-13T16:50:35.501389+0000","last_fullsized":"2024-09-13T16:50:35.501389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:58.754567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463122+0000","last_change":"2024-09-13T16:50:30.614804+0000","last_active":"2024-09-13T16:50:35.463122+0000","last_peered":"2024-09-13T16:50:35.463122+0000","last_clean":"2024-09-13T16:50:35.463122+0000","last_became_active":"2024-09-13T16:50:30.614458+0000","last_became_peered":"2024-09-13T16:50:30.614458+0000","last_unstale":"2024-09-13T16:50:35.463122+0000","last_undegraded":"2024-09-13T16:50:35.463122+0000","last_fullsized":"2024-09-13T16:50:35.463122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093301+0000","last_change":"2024-09-13T16:50:30.107253+0000","last_active":"2024-09-13T16:50:35.093301+0000","last_peered":"2024-09-13T16:50:35.093301+0000","last_clean":"2024-09-13T16:50:35.093301+0000","last_became_active":"2024-09-13T16:50:30.106448+0000","last_became_peered":"2024-09-13T16:50:30.106448+0000","last_unstale":"2024-09-13T16:50:35.093301+0000","last_undegraded":"2024-09-13T16:50:35.093301+0000","last_fullsized":"2024-09-13T16:50:35.093301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.507157+0000","last_change":"2024-09-13T16:50:35.504918+0000","last_active":"2024-09-13T16:50:35.507157+0000","last_peered":"2024-09-13T16:50:35.507157+0000","last_clean":"2024-09-13T16:50:35.507157+0000","last_became_active":"2024-09-13T16:50:35.504252+0000","last_became_peered":"2024-09-13T16:50:35.504252+0000","last_unstale":"2024-09-13T16:50:35.507157+0000","last_undegraded":"2024-09-13T16:50:35.507157+0000","last_fullsized":"2024-09-13T16:50:35.507157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:54.702371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456826+0000","last_change":"2024-09-13T16:50:30.610870+0000","last_active":"2024-09-13T16:50:35.456826+0000","last_peered":"2024-09-13T16:50:35.456826+0000","last_clean":"2024-09-13T16:50:35.456826+0000","last_became_active":"2024-09-13T16:50:30.610473+0000","last_became_peered":"2024-09-13T16:50:30.610473+0000","last_unstale":"2024-09-13T16:50:35.456826+0000","last_undegraded":"2024-09-13T16:50:35.456826+0000","last_fullsized":"2024-09-13T16:50:35.456826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.940724+0000","last_change":"2024-09-13T16:50:30.111250+0000","last_active":"2024-09-13T16:50:35.940724+0000","last_peered":"2024-09-13T16:50:35.940724+0000","last_clean":"2024-09-13T16:50:35.940724+0000","last_became_active":"2024-09-13T16:50:30.109959+0000","last_became_peered":"2024-09-13T16:50:30.109959+0000","last_unstale":"2024-09-13T16:50:35.940724+0000","last_undegraded":"2024-09-13T16:50:35.940724+0000","last_fullsized":"2024-09-13T16:50:35.940724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509887+0000","last_change":"2024-09-13T16:50:35.501491+0000","last_active":"2024-09-13T16:50:35.509887+0000","last_peered":"2024-09-13T16:50:35.509887+0000","last_clean":"2024-09-13T16:50:35.509887+0000","last_became_active":"2024-09-13T16:50:35.500108+0000","last_became_peered":"2024-09-13T16:50:35.500108+0000","last_unstale":"2024-09-13T16:50:35.509887+0000","last_undegraded":"2024-09-13T16:50:35.509887+0000","last_fullsized":"2024-09-13T16:50:35.509887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:30.666178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401804+0000","last_change":"2024-09-13T16:50:30.606888+0000","last_active":"2024-09-13T16:50:35.401804+0000","last_peered":"2024-09-13T16:50:35.401804+0000","last_clean":"2024-09-13T16:50:35.401804+0000","last_became_active":"2024-09-13T16:50:30.605360+0000","last_became_peered":"2024-09-13T16:50:30.605360+0000","last_unstale":"2024-09-13T16:50:35.401804+0000","last_undegraded":"2024-09-13T16:50:35.401804+0000","last_fullsized":"2024-09-13T16:50:35.401804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500383+0000","last_change":"2024-09-13T16:50:30.111119+0000","last_active":"2024-09-13T16:50:35.500383+0000","last_peered":"2024-09-13T16:50:35.500383+0000","last_clean":"2024-09-13T16:50:35.500383+0000","last_became_active":"2024-09-13T16:50:30.110892+0000","last_became_peered":"2024-09-13T16:50:30.110892+0000","last_unstale":"2024-09-13T16:50:35.500383+0000","last_undegraded":"2024-09-13T16:50:35.500383+0000","last_fullsized":"2024-09-13T16:50:35.500383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509685+0000","last_change":"2024-09-13T16:50:35.505149+0000","last_active":"2024-09-13T16:50:35.509685+0000","last_peered":"2024-09-13T16:50:35.509685+0000","last_clean":"2024-09-13T16:50:35.509685+0000","last_became_active":"2024-09-13T16:50:35.502086+0000","last_became_peered":"2024-09-13T16:50:35.502086+0000","last_unstale":"2024-09-13T16:50:35.509685+0000","last_undegraded":"2024-09-13T16:50:35.509685+0000","last_fullsized":"2024-09-13T16:50:35.509685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092835+0000","last_change":"2024-09-13T16:50:30.601700+0000","last_active":"2024-09-13T16:50:35.092835+0000","last_peered":"2024-09-13T16:50:35.092835+0000","last_clean":"2024-09-13T16:50:35.092835+0000","last_became_active":"2024-09-13T16:50:30.601432+0000","last_became_peered":"2024-09-13T16:50:30.601432+0000","last_unstale":"2024-09-13T16:50:35.092835+0000","last_undegraded":"2024-09-13T16:50:35.092835+0000","last_fullsized":"2024-09-13T16:50:35.092835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:25.758134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402007+0000","last_change":"2024-09-13T16:50:30.112636+0000","last_active":"2024-09-13T16:50:35.402007+0000","last_peered":"2024-09-13T16:50:35.402007+0000","last_clean":"2024-09-13T16:50:35.402007+0000","last_became_active":"2024-09-13T16:50:30.112121+0000","last_became_peered":"2024-09-13T16:50:30.112121+0000","last_unstale":"2024-09-13T16:50:35.402007+0000","last_undegraded":"2024-09-13T16:50:35.402007+0000","last_fullsized":"2024-09-13T16:50:35.402007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401192+0000","last_change":"2024-09-13T16:50:34.633860+0000","last_active":"2024-09-13T16:50:35.401192+0000","last_peered":"2024-09-13T16:50:35.401192+0000","last_clean":"2024-09-13T16:50:35.401192+0000","last_became_active":"2024-09-13T16:50:34.633435+0000","last_became_peered":"2024-09-13T16:50:34.633435+0000","last_unstale":"2024-09-13T16:50:35.401192+0000","last_undegraded":"2024-09-13T16:50:35.401192+0000","last_fullsized":"2024-09-13T16:50:35.401192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:27.883765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501232+0000","last_change":"2024-09-13T16:50:30.613267+0000","last_active":"2024-09-13T16:50:35.501232+0000","last_peered":"2024-09-13T16:50:35.501232+0000","last_clean":"2024-09-13T16:50:35.501232+0000","last_became_active":"2024-09-13T16:50:30.612861+0000","last_became_peered":"2024-09-13T16:50:30.612861+0000","last_unstale":"2024-09-13T16:50:35.501232+0000","last_undegraded":"2024-09-13T16:50:35.501232+0000","last_fullsized":"2024-09-13T16:50:35.501232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:27.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091946+0000","last_change":"2024-09-13T16:50:30.110348+0000","last_active":"2024-09-13T16:50:35.091946+0000","last_peered":"2024-09-13T16:50:35.091946+0000","last_clean":"2024-09-13T16:50:35.091946+0000","last_became_active":"2024-09-13T16:50:30.110041+0000","last_became_peered":"2024-09-13T16:50:30.110041+0000","last_unstale":"2024-09-13T16:50:35.091946+0000","last_undegraded":"2024-09-13T16:50:35.091946+0000","last_fullsized":"2024-09-13T16:50:35.091946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401674+0000","last_change":"2024-09-13T16:50:34.633493+0000","last_active":"2024-09-13T16:50:35.401674+0000","last_peered":"2024-09-13T16:50:35.401674+0000","last_clean":"2024-09-13T16:50:35.401674+0000","last_became_active":"2024-09-13T16:50:34.632442+0000","last_became_peered":"2024-09-13T16:50:34.632442+0000","last_unstale":"2024-09-13T16:50:35.401674+0000","last_undegraded":"2024-09-13T16:50:35.401674+0000","last_fullsized":"2024-09-13T16:50:35.401674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401509+0000","last_change":"2024-09-13T16:50:30.590118+0000","last_active":"2024-09-13T16:50:35.401509+0000","last_peered":"2024-09-13T16:50:35.401509+0000","last_clean":"2024-09-13T16:50:35.401509+0000","last_became_active":"2024-09-13T16:50:30.589880+0000","last_became_peered":"2024-09-13T16:50:30.589880+0000","last_unstale":"2024-09-13T16:50:35.401509+0000","last_undegraded":"2024-09-13T16:50:35.401509+0000","last_fullsized":"2024-09-13T16:50:35.401509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:32.908616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087743+0000","last_change":"2024-09-13T16:50:30.103427+0000","last_active":"2024-09-13T16:50:35.087743+0000","last_peered":"2024-09-13T16:50:35.087743+0000","last_clean":"2024-09-13T16:50:35.087743+0000","last_became_active":"2024-09-13T16:50:30.102964+0000","last_became_peered":"2024-09-13T16:50:30.102964+0000","last_unstale":"2024-09-13T16:50:35.087743+0000","last_undegraded":"2024-09-13T16:50:35.087743+0000","last_fullsized":"2024-09-13T16:50:35.087743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402109+0000","last_change":"2024-09-13T16:50:34.632823+0000","last_active":"2024-09-13T16:50:35.402109+0000","last_peered":"2024-09-13T16:50:35.402109+0000","last_clean":"2024-09-13T16:50:35.402109+0000","last_became_active":"2024-09-13T16:50:34.632261+0000","last_became_peered":"2024-09-13T16:50:34.632261+0000","last_unstale":"2024-09-13T16:50:35.402109+0000","last_undegraded":"2024-09-13T16:50:35.402109+0000","last_fullsized":"2024-09-13T16:50:35.402109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462419+0000","last_change":"2024-09-13T16:50:30.606638+0000","last_active":"2024-09-13T16:50:35.462419+0000","last_peered":"2024-09-13T16:50:35.462419+0000","last_clean":"2024-09-13T16:50:35.462419+0000","last_became_active":"2024-09-13T16:50:30.606375+0000","last_became_peered":"2024-09-13T16:50:30.606375+0000","last_unstale":"2024-09-13T16:50:35.462419+0000","last_undegraded":"2024-09-13T16:50:35.462419+0000","last_fullsized":"2024-09-13T16:50:35.462419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:27.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498015+0000","last_change":"2024-09-13T16:50:30.115629+0000","last_active":"2024-09-13T16:50:35.498015+0000","last_peered":"2024-09-13T16:50:35.498015+0000","last_clean":"2024-09-13T16:50:35.498015+0000","last_became_active":"2024-09-13T16:50:30.115336+0000","last_became_peered":"2024-09-13T16:50:30.115336+0000","last_unstale":"2024-09-13T16:50:35.498015+0000","last_undegraded":"2024-09-13T16:50:35.498015+0000","last_fullsized":"2024-09-13T16:50:35.498015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:47.419945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509728+0000","last_change":"2024-09-13T16:50:35.507866+0000","last_active":"2024-09-13T16:50:35.509728+0000","last_peered":"2024-09-13T16:50:35.509728+0000","last_clean":"2024-09-13T16:50:35.509728+0000","last_became_active":"2024-09-13T16:50:35.505548+0000","last_became_peered":"2024-09-13T16:50:35.505548+0000","last_unstale":"2024-09-13T16:50:35.509728+0000","last_undegraded":"2024-09-13T16:50:35.509728+0000","last_fullsized":"2024-09-13T16:50:35.509728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.455646+0000","last_change":"2024-09-13T16:50:30.596915+0000","last_active":"2024-09-13T16:50:35.455646+0000","last_peered":"2024-09-13T16:50:35.455646+0000","last_clean":"2024-09-13T16:50:35.455646+0000","last_became_active":"2024-09-13T16:50:30.596643+0000","last_became_peered":"2024-09-13T16:50:30.596643+0000","last_unstale":"2024-09-13T16:50:35.455646+0000","last_undegraded":"2024-09-13T16:50:35.455646+0000","last_fullsized":"2024-09-13T16:50:35.455646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092229+0000","last_change":"2024-09-13T16:50:30.115172+0000","last_active":"2024-09-13T16:50:35.092229+0000","last_peered":"2024-09-13T16:50:35.092229+0000","last_clean":"2024-09-13T16:50:35.092229+0000","last_became_active":"2024-09-13T16:50:30.114633+0000","last_became_peered":"2024-09-13T16:50:30.114633+0000","last_unstale":"2024-09-13T16:50:35.092229+0000","last_undegraded":"2024-09-13T16:50:35.092229+0000","last_fullsized":"2024-09-13T16:50:35.092229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498080+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:50:35.498080+0000","last_peered":"2024-09-13T16:50:35.498080+0000","last_clean":"2024-09-13T16:50:35.498080+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:50:35.498080+0000","last_undegraded":"2024-09-13T16:50:35.498080+0000","last_fullsized":"2024-09-13T16:50:35.498080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.506121+0000","last_change":"2024-09-13T16:50:35.502219+0000","last_active":"2024-09-13T16:50:35.506121+0000","last_peered":"2024-09-13T16:50:35.506121+0000","last_clean":"2024-09-13T16:50:35.506121+0000","last_became_active":"2024-09-13T16:50:35.501877+0000","last_became_peered":"2024-09-13T16:50:35.501877+0000","last_unstale":"2024-09-13T16:50:35.506121+0000","last_undegraded":"2024-09-13T16:50:35.506121+0000","last_fullsized":"2024-09-13T16:50:35.506121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:37.440533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500614+0000","last_change":"2024-09-13T16:50:30.604143+0000","last_active":"2024-09-13T16:50:35.500614+0000","last_peered":"2024-09-13T16:50:35.500614+0000","last_clean":"2024-09-13T16:50:35.500614+0000","last_became_active":"2024-09-13T16:50:30.603882+0000","last_became_peered":"2024-09-13T16:50:30.603882+0000","last_unstale":"2024-09-13T16:50:35.500614+0000","last_undegraded":"2024-09-13T16:50:35.500614+0000","last_fullsized":"2024-09-13T16:50:35.500614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:31.996725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087857+0000","last_change":"2024-09-13T16:50:30.116691+0000","last_active":"2024-09-13T16:50:35.087857+0000","last_peered":"2024-09-13T16:50:35.087857+0000","last_clean":"2024-09-13T16:50:35.087857+0000","last_became_active":"2024-09-13T16:50:30.115262+0000","last_became_peered":"2024-09-13T16:50:30.115262+0000","last_unstale":"2024-09-13T16:50:35.087857+0000","last_undegraded":"2024-09-13T16:50:35.087857+0000","last_fullsized":"2024-09-13T16:50:35.087857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.506161+0000","last_change":"2024-09-13T16:50:35.499612+0000","last_active":"2024-09-13T16:50:35.506161+0000","last_peered":"2024-09-13T16:50:35.506161+0000","last_clean":"2024-09-13T16:50:35.506161+0000","last_became_active":"2024-09-13T16:50:35.498882+0000","last_became_peered":"2024-09-13T16:50:35.498882+0000","last_unstale":"2024-09-13T16:50:35.506161+0000","last_undegraded":"2024-09-13T16:50:35.506161+0000","last_fullsized":"2024-09-13T16:50:35.506161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:14.360209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092696+0000","last_change":"2024-09-13T16:50:30.601337+0000","last_active":"2024-09-13T16:50:35.092696+0000","last_peered":"2024-09-13T16:50:35.092696+0000","last_clean":"2024-09-13T16:50:35.092696+0000","last_became_active":"2024-09-13T16:50:30.601073+0000","last_became_peered":"2024-09-13T16:50:30.601073+0000","last_unstale":"2024-09-13T16:50:35.092696+0000","last_undegraded":"2024-09-13T16:50:35.092696+0000","last_fullsized":"2024-09-13T16:50:35.092696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.117003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402285+0000","last_change":"2024-09-13T16:50:30.109801+0000","last_active":"2024-09-13T16:50:35.402285+0000","last_peered":"2024-09-13T16:50:35.402285+0000","last_clean":"2024-09-13T16:50:35.402285+0000","last_became_active":"2024-09-13T16:50:30.107936+0000","last_became_peered":"2024-09-13T16:50:30.107936+0000","last_unstale":"2024-09-13T16:50:35.402285+0000","last_undegraded":"2024-09-13T16:50:35.402285+0000","last_fullsized":"2024-09-13T16:50:35.402285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462892+0000","last_change":"2024-09-13T16:50:34.640936+0000","last_active":"2024-09-13T16:50:35.462892+0000","last_peered":"2024-09-13T16:50:35.462892+0000","last_clean":"2024-09-13T16:50:35.462892+0000","last_became_active":"2024-09-13T16:50:34.640606+0000","last_became_peered":"2024-09-13T16:50:34.640606+0000","last_unstale":"2024-09-13T16:50:35.462892+0000","last_undegraded":"2024-09-13T16:50:35.462892+0000","last_fullsized":"2024-09-13T16:50:35.462892+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500787+0000","last_change":"2024-09-13T16:50:30.620140+0000","last_active":"2024-09-13T16:50:35.500787+0000","last_peered":"2024-09-13T16:50:35.500787+0000","last_clean":"2024-09-13T16:50:35.500787+0000","last_became_active":"2024-09-13T16:50:30.619960+0000","last_became_peered":"2024-09-13T16:50:30.619960+0000","last_unstale":"2024-09-13T16:50:35.500787+0000","last_undegraded":"2024-09-13T16:50:35.500787+0000","last_fullsized":"2024-09-13T16:50:35.500787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:28.715573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.088162+0000","last_change":"2024-09-13T16:50:30.104959+0000","last_active":"2024-09-13T16:50:35.088162+0000","last_peered":"2024-09-13T16:50:35.088162+0000","last_clean":"2024-09-13T16:50:35.088162+0000","last_became_active":"2024-09-13T16:50:30.104709+0000","last_became_peered":"2024-09-13T16:50:30.104709+0000","last_unstale":"2024-09-13T16:50:35.088162+0000","last_undegraded":"2024-09-13T16:50:35.088162+0000","last_fullsized":"2024-09-13T16:50:35.088162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.828286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.505477+0000","last_change":"2024-09-13T16:50:35.501120+0000","last_active":"2024-09-13T16:50:35.505477+0000","last_peered":"2024-09-13T16:50:35.505477+0000","last_clean":"2024-09-13T16:50:35.505477+0000","last_became_active":"2024-09-13T16:50:35.500500+0000","last_became_peered":"2024-09-13T16:50:35.500500+0000","last_unstale":"2024-09-13T16:50:35.505477+0000","last_undegraded":"2024-09-13T16:50:35.505477+0000","last_fullsized":"2024-09-13T16:50:35.505477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499025+0000","last_change":"2024-09-13T16:50:30.615966+0000","last_active":"2024-09-13T16:50:35.499025+0000","last_peered":"2024-09-13T16:50:35.499025+0000","last_clean":"2024-09-13T16:50:35.499025+0000","last_became_active":"2024-09-13T16:50:30.615750+0000","last_became_peered":"2024-09-13T16:50:30.615750+0000","last_unstale":"2024-09-13T16:50:35.499025+0000","last_undegraded":"2024-09-13T16:50:35.499025+0000","last_fullsized":"2024-09-13T16:50:35.499025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501690+0000","last_change":"2024-09-13T16:50:30.112955+0000","last_active":"2024-09-13T16:50:35.501690+0000","last_peered":"2024-09-13T16:50:35.501690+0000","last_clean":"2024-09-13T16:50:35.501690+0000","last_became_active":"2024-09-13T16:50:30.111633+0000","last_became_peered":"2024-09-13T16:50:30.111633+0000","last_unstale":"2024-09-13T16:50:35.501690+0000","last_undegraded":"2024-09-13T16:50:35.501690+0000","last_fullsized":"2024-09-13T16:50:35.501690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499166+0000","last_change":"2024-09-13T16:50:34.636875+0000","last_active":"2024-09-13T16:50:35.499166+0000","last_peered":"2024-09-13T16:50:35.499166+0000","last_clean":"2024-09-13T16:50:35.499166+0000","last_became_active":"2024-09-13T16:50:34.636350+0000","last_became_peered":"2024-09-13T16:50:34.636350+0000","last_unstale":"2024-09-13T16:50:35.499166+0000","last_undegraded":"2024-09-13T16:50:35.499166+0000","last_fullsized":"2024-09-13T16:50:35.499166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:32.980957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498929+0000","last_change":"2024-09-13T16:50:30.617707+0000","last_active":"2024-09-13T16:50:35.498929+0000","last_peered":"2024-09-13T16:50:35.498929+0000","last_clean":"2024-09-13T16:50:35.498929+0000","last_became_active":"2024-09-13T16:50:30.617279+0000","last_became_peered":"2024-09-13T16:50:30.617279+0000","last_unstale":"2024-09-13T16:50:35.498929+0000","last_undegraded":"2024-09-13T16:50:35.498929+0000","last_fullsized":"2024-09-13T16:50:35.498929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087386+0000","last_change":"2024-09-13T16:50:30.107834+0000","last_active":"2024-09-13T16:50:35.087386+0000","last_peered":"2024-09-13T16:50:35.087386+0000","last_clean":"2024-09-13T16:50:35.087386+0000","last_became_active":"2024-09-13T16:50:30.107457+0000","last_became_peered":"2024-09-13T16:50:30.107457+0000","last_unstale":"2024-09-13T16:50:35.087386+0000","last_undegraded":"2024-09-13T16:50:35.087386+0000","last_fullsized":"2024-09-13T16:50:35.087386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463011+0000","last_change":"2024-09-13T16:50:34.635554+0000","last_active":"2024-09-13T16:50:35.463011+0000","last_peered":"2024-09-13T16:50:35.463011+0000","last_clean":"2024-09-13T16:50:35.463011+0000","last_became_active":"2024-09-13T16:50:34.635194+0000","last_became_peered":"2024-09-13T16:50:34.635194+0000","last_unstale":"2024-09-13T16:50:35.463011+0000","last_undegraded":"2024-09-13T16:50:35.463011+0000","last_fullsized":"2024-09-13T16:50:35.463011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:54.702371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091123+0000","last_change":"2024-09-13T16:50:30.612793+0000","last_active":"2024-09-13T16:50:35.091123+0000","last_peered":"2024-09-13T16:50:35.091123+0000","last_clean":"2024-09-13T16:50:35.091123+0000","last_became_active":"2024-09-13T16:50:30.612243+0000","last_became_peered":"2024-09-13T16:50:30.612243+0000","last_unstale":"2024-09-13T16:50:35.091123+0000","last_undegraded":"2024-09-13T16:50:35.091123+0000","last_fullsized":"2024-09-13T16:50:35.091123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091094+0000","last_change":"2024-09-13T16:50:30.097963+0000","last_active":"2024-09-13T16:50:35.091094+0000","last_peered":"2024-09-13T16:50:35.091094+0000","last_clean":"2024-09-13T16:50:35.091094+0000","last_became_active":"2024-09-13T16:50:30.097015+0000","last_became_peered":"2024-09-13T16:50:30.097015+0000","last_unstale":"2024-09-13T16:50:35.091094+0000","last_undegraded":"2024-09-13T16:50:35.091094+0000","last_fullsized":"2024-09-13T16:50:35.091094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.514416+0000","last_change":"2024-09-13T16:50:35.512427+0000","last_active":"2024-09-13T16:50:35.514416+0000","last_peered":"2024-09-13T16:50:35.514416+0000","last_clean":"2024-09-13T16:50:35.514416+0000","last_became_active":"2024-09-13T16:50:35.512074+0000","last_became_peered":"2024-09-13T16:50:35.512074+0000","last_unstale":"2024-09-13T16:50:35.514416+0000","last_undegraded":"2024-09-13T16:50:35.514416+0000","last_fullsized":"2024-09-13T16:50:35.514416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:37.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498993+0000","last_change":"2024-09-13T16:50:30.617270+0000","last_active":"2024-09-13T16:50:35.498993+0000","last_peered":"2024-09-13T16:50:35.498993+0000","last_clean":"2024-09-13T16:50:35.498993+0000","last_became_active":"2024-09-13T16:50:30.617047+0000","last_became_peered":"2024-09-13T16:50:30.617047+0000","last_unstale":"2024-09-13T16:50:35.498993+0000","last_undegraded":"2024-09-13T16:50:35.498993+0000","last_fullsized":"2024-09-13T16:50:35.498993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091892+0000","last_change":"2024-09-13T16:50:30.109703+0000","last_active":"2024-09-13T16:50:35.091892+0000","last_peered":"2024-09-13T16:50:35.091892+0000","last_clean":"2024-09-13T16:50:35.091892+0000","last_became_active":"2024-09-13T16:50:30.109427+0000","last_became_peered":"2024-09-13T16:50:30.109427+0000","last_unstale":"2024-09-13T16:50:35.091892+0000","last_undegraded":"2024-09-13T16:50:35.091892+0000","last_fullsized":"2024-09-13T16:50:35.091892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509958+0000","last_change":"2024-09-13T16:50:35.501616+0000","last_active":"2024-09-13T16:50:35.509958+0000","last_peered":"2024-09-13T16:50:35.509958+0000","last_clean":"2024-09-13T16:50:35.509958+0000","last_became_active":"2024-09-13T16:50:35.500854+0000","last_became_peered":"2024-09-13T16:50:35.500854+0000","last_unstale":"2024-09-13T16:50:35.509958+0000","last_undegraded":"2024-09-13T16:50:35.509958+0000","last_fullsized":"2024-09-13T16:50:35.509958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:18.549426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462681+0000","last_change":"2024-09-13T16:50:30.619507+0000","last_active":"2024-09-13T16:50:35.462681+0000","last_peered":"2024-09-13T16:50:35.462681+0000","last_clean":"2024-09-13T16:50:35.462681+0000","last_became_active":"2024-09-13T16:50:30.619122+0000","last_became_peered":"2024-09-13T16:50:30.619122+0000","last_unstale":"2024-09-13T16:50:35.462681+0000","last_undegraded":"2024-09-13T16:50:35.462681+0000","last_fullsized":"2024-09-13T16:50:35.462681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:00.203031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463139+0000","last_change":"2024-09-13T16:50:30.099206+0000","last_active":"2024-09-13T16:50:35.463139+0000","last_peered":"2024-09-13T16:50:35.463139+0000","last_clean":"2024-09-13T16:50:35.463139+0000","last_became_active":"2024-09-13T16:50:30.098688+0000","last_became_peered":"2024-09-13T16:50:30.098688+0000","last_unstale":"2024-09-13T16:50:35.463139+0000","last_undegraded":"2024-09-13T16:50:35.463139+0000","last_fullsized":"2024-09-13T16:50:35.463139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.505292+0000","last_change":"2024-09-13T16:50:35.502406+0000","last_active":"2024-09-13T16:50:35.505292+0000","last_peered":"2024-09-13T16:50:35.505292+0000","last_clean":"2024-09-13T16:50:35.505292+0000","last_became_active":"2024-09-13T16:50:35.502014+0000","last_became_peered":"2024-09-13T16:50:35.502014+0000","last_unstale":"2024-09-13T16:50:35.505292+0000","last_undegraded":"2024-09-13T16:50:35.505292+0000","last_fullsized":"2024-09-13T16:50:35.505292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498570+0000","last_change":"2024-09-13T16:50:30.613053+0000","last_active":"2024-09-13T16:50:35.498570+0000","last_peered":"2024-09-13T16:50:35.498570+0000","last_clean":"2024-09-13T16:50:35.498570+0000","last_became_active":"2024-09-13T16:50:30.612775+0000","last_became_peered":"2024-09-13T16:50:30.612775+0000","last_unstale":"2024-09-13T16:50:35.498570+0000","last_undegraded":"2024-09-13T16:50:35.498570+0000","last_fullsized":"2024-09-13T16:50:35.498570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:54.244738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T16:50:47.108380+0000","last_change":"2024-09-13T16:50:30.111095+0000","last_active":"2024-09-13T16:50:47.108380+0000","last_peered":"2024-09-13T16:50:47.108380+0000","last_clean":"2024-09-13T16:50:47.108380+0000","last_became_active":"2024-09-13T16:50:30.110886+0000","last_became_peered":"2024-09-13T16:50:30.110886+0000","last_unstale":"2024-09-13T16:50:47.108380+0000","last_undegraded":"2024-09-13T16:50:47.108380+0000","last_fullsized":"2024-09-13T16:50:47.108380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401424+0000","last_change":"2024-09-13T16:50:34.641283+0000","last_active":"2024-09-13T16:50:35.401424+0000","last_peered":"2024-09-13T16:50:35.401424+0000","last_clean":"2024-09-13T16:50:35.401424+0000","last_became_active":"2024-09-13T16:50:34.636369+0000","last_became_peered":"2024-09-13T16:50:34.636369+0000","last_unstale":"2024-09-13T16:50:35.401424+0000","last_undegraded":"2024-09-13T16:50:35.401424+0000","last_fullsized":"2024-09-13T16:50:35.401424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:46.586048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.502226+0000","last_change":"2024-09-13T16:50:30.611541+0000","last_active":"2024-09-13T16:50:35.502226+0000","last_peered":"2024-09-13T16:50:35.502226+0000","last_clean":"2024-09-13T16:50:35.502226+0000","last_became_active":"2024-09-13T16:50:30.611046+0000","last_became_peered":"2024-09-13T16:50:30.611046+0000","last_unstale":"2024-09-13T16:50:35.502226+0000","last_undegraded":"2024-09-13T16:50:35.502226+0000","last_fullsized":"2024-09-13T16:50:35.502226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:51.423821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402184+0000","last_change":"2024-09-13T16:50:30.111117+0000","last_active":"2024-09-13T16:50:35.402184+0000","last_peered":"2024-09-13T16:50:35.402184+0000","last_clean":"2024-09-13T16:50:35.402184+0000","last_became_active":"2024-09-13T16:50:30.110857+0000","last_became_peered":"2024-09-13T16:50:30.110857+0000","last_unstale":"2024-09-13T16:50:35.402184+0000","last_undegraded":"2024-09-13T16:50:35.402184+0000","last_fullsized":"2024-09-13T16:50:35.402184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500960+0000","last_change":"2024-09-13T16:50:34.634925+0000","last_active":"2024-09-13T16:50:35.500960+0000","last_peered":"2024-09-13T16:50:35.500960+0000","last_clean":"2024-09-13T16:50:35.500960+0000","last_became_active":"2024-09-13T16:50:34.634329+0000","last_became_peered":"2024-09-13T16:50:34.634329+0000","last_unstale":"2024-09-13T16:50:35.500960+0000","last_undegraded":"2024-09-13T16:50:35.500960+0000","last_fullsized":"2024-09-13T16:50:35.500960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:30.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463375+0000","last_change":"2024-09-13T16:50:30.609227+0000","last_active":"2024-09-13T16:50:35.463375+0000","last_peered":"2024-09-13T16:50:35.463375+0000","last_clean":"2024-09-13T16:50:35.463375+0000","last_became_active":"2024-09-13T16:50:30.608863+0000","last_became_peered":"2024-09-13T16:50:30.608863+0000","last_unstale":"2024-09-13T16:50:35.463375+0000","last_undegraded":"2024-09-13T16:50:35.463375+0000","last_fullsized":"2024-09-13T16:50:35.463375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093039+0000","last_change":"2024-09-13T16:50:30.109198+0000","last_active":"2024-09-13T16:50:35.093039+0000","last_peered":"2024-09-13T16:50:35.093039+0000","last_clean":"2024-09-13T16:50:35.093039+0000","last_became_active":"2024-09-13T16:50:30.108896+0000","last_became_peered":"2024-09-13T16:50:30.108896+0000","last_unstale":"2024-09-13T16:50:35.093039+0000","last_undegraded":"2024-09-13T16:50:35.093039+0000","last_fullsized":"2024-09-13T16:50:35.093039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456883+0000","last_change":"2024-09-13T16:50:34.630350+0000","last_active":"2024-09-13T16:50:35.456883+0000","last_peered":"2024-09-13T16:50:35.456883+0000","last_clean":"2024-09-13T16:50:35.456883+0000","last_became_active":"2024-09-13T16:50:34.626215+0000","last_became_peered":"2024-09-13T16:50:34.626215+0000","last_unstale":"2024-09-13T16:50:35.456883+0000","last_undegraded":"2024-09-13T16:50:35.456883+0000","last_fullsized":"2024-09-13T16:50:35.456883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:51.705201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092550+0000","last_change":"2024-09-13T16:50:30.603528+0000","last_active":"2024-09-13T16:50:35.092550+0000","last_peered":"2024-09-13T16:50:35.092550+0000","last_clean":"2024-09-13T16:50:35.092550+0000","last_became_active":"2024-09-13T16:50:30.603265+0000","last_became_peered":"2024-09-13T16:50:30.603265+0000","last_unstale":"2024-09-13T16:50:35.092550+0000","last_undegraded":"2024-09-13T16:50:35.092550+0000","last_fullsized":"2024-09-13T16:50:35.092550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093205+0000","last_change":"2024-09-13T16:50:30.112126+0000","last_active":"2024-09-13T16:50:35.093205+0000","last_peered":"2024-09-13T16:50:35.093205+0000","last_clean":"2024-09-13T16:50:35.093205+0000","last_became_active":"2024-09-13T16:50:30.111844+0000","last_became_peered":"2024-09-13T16:50:30.111844+0000","last_unstale":"2024-09-13T16:50:35.093205+0000","last_undegraded":"2024-09-13T16:50:35.093205+0000","last_fullsized":"2024-09-13T16:50:35.093205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.511284+0000","last_change":"2024-09-13T16:50:35.507739+0000","last_active":"2024-09-13T16:50:35.511284+0000","last_peered":"2024-09-13T16:50:35.511284+0000","last_clean":"2024-09-13T16:50:35.511284+0000","last_became_active":"2024-09-13T16:50:35.505030+0000","last_became_peered":"2024-09-13T16:50:35.505030+0000","last_unstale":"2024-09-13T16:50:35.511284+0000","last_undegraded":"2024-09-13T16:50:35.511284+0000","last_fullsized":"2024-09-13T16:50:35.511284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:29.316344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509712+0000","last_change":"2024-09-13T16:50:35.506793+0000","last_active":"2024-09-13T16:50:35.509712+0000","last_peered":"2024-09-13T16:50:35.509712+0000","last_clean":"2024-09-13T16:50:35.509712+0000","last_became_active":"2024-09-13T16:50:35.505380+0000","last_became_peered":"2024-09-13T16:50:35.505380+0000","last_unstale":"2024-09-13T16:50:35.509712+0000","last_undegraded":"2024-09-13T16:50:35.509712+0000","last_fullsized":"2024-09-13T16:50:35.509712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:58.754567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.400877+0000","last_change":"2024-09-13T16:50:30.601660+0000","last_active":"2024-09-13T16:50:35.400877+0000","last_peered":"2024-09-13T16:50:35.400877+0000","last_clean":"2024-09-13T16:50:35.400877+0000","last_became_active":"2024-09-13T16:50:30.601315+0000","last_became_peered":"2024-09-13T16:50:30.601315+0000","last_unstale":"2024-09-13T16:50:35.400877+0000","last_undegraded":"2024-09-13T16:50:35.400877+0000","last_fullsized":"2024-09-13T16:50:35.400877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091503+0000","last_change":"2024-09-13T16:50:30.098656+0000","last_active":"2024-09-13T16:50:35.091503+0000","last_peered":"2024-09-13T16:50:35.091503+0000","last_clean":"2024-09-13T16:50:35.091503+0000","last_became_active":"2024-09-13T16:50:30.098137+0000","last_became_peered":"2024-09-13T16:50:30.098137+0000","last_unstale":"2024-09-13T16:50:35.091503+0000","last_undegraded":"2024-09-13T16:50:35.091503+0000","last_fullsized":"2024-09-13T16:50:35.091503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093095+0000","last_change":"2024-09-13T16:50:34.632360+0000","last_active":"2024-09-13T16:50:35.093095+0000","last_peered":"2024-09-13T16:50:35.093095+0000","last_clean":"2024-09-13T16:50:35.093095+0000","last_became_active":"2024-09-13T16:50:34.631958+0000","last_became_peered":"2024-09-13T16:50:34.631958+0000","last_unstale":"2024-09-13T16:50:35.093095+0000","last_undegraded":"2024-09-13T16:50:35.093095+0000","last_fullsized":"2024-09-13T16:50:35.093095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:30.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456997+0000","last_change":"2024-09-13T16:50:30.609936+0000","last_active":"2024-09-13T16:50:35.456997+0000","last_peered":"2024-09-13T16:50:35.456997+0000","last_clean":"2024-09-13T16:50:35.456997+0000","last_became_active":"2024-09-13T16:50:30.609683+0000","last_became_peered":"2024-09-13T16:50:30.609683+0000","last_unstale":"2024-09-13T16:50:35.456997+0000","last_undegraded":"2024-09-13T16:50:35.456997+0000","last_fullsized":"2024-09-13T16:50:35.456997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:32.908616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456931+0000","last_change":"2024-09-13T16:50:30.103999+0000","last_active":"2024-09-13T16:50:35.456931+0000","last_peered":"2024-09-13T16:50:35.456931+0000","last_clean":"2024-09-13T16:50:35.456931+0000","last_became_active":"2024-09-13T16:50:30.103531+0000","last_became_peered":"2024-09-13T16:50:30.103531+0000","last_unstale":"2024-09-13T16:50:35.456931+0000","last_undegraded":"2024-09-13T16:50:35.456931+0000","last_fullsized":"2024-09-13T16:50:35.456931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.828286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091972+0000","last_change":"2024-09-13T16:50:30.103767+0000","last_active":"2024-09-13T16:50:35.091972+0000","last_peered":"2024-09-13T16:50:35.091972+0000","last_clean":"2024-09-13T16:50:35.091972+0000","last_became_active":"2024-09-13T16:50:30.103519+0000","last_became_peered":"2024-09-13T16:50:30.103519+0000","last_unstale":"2024-09-13T16:50:35.091972+0000","last_undegraded":"2024-09-13T16:50:35.091972+0000","last_fullsized":"2024-09-13T16:50:35.091972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463060+0000","last_change":"2024-09-13T16:50:30.618557+0000","last_active":"2024-09-13T16:50:35.463060+0000","last_peered":"2024-09-13T16:50:35.463060+0000","last_clean":"2024-09-13T16:50:35.463060+0000","last_became_active":"2024-09-13T16:50:30.618308+0000","last_became_peered":"2024-09-13T16:50:30.618308+0000","last_unstale":"2024-09-13T16:50:35.463060+0000","last_undegraded":"2024-09-13T16:50:35.463060+0000","last_fullsized":"2024-09-13T16:50:35.463060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:45.160388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087606+0000","last_change":"2024-09-13T16:50:34.639885+0000","last_active":"2024-09-13T16:50:35.087606+0000","last_peered":"2024-09-13T16:50:35.087606+0000","last_clean":"2024-09-13T16:50:35.087606+0000","last_became_active":"2024-09-13T16:50:34.639399+0000","last_became_peered":"2024-09-13T16:50:34.639399+0000","last_unstale":"2024-09-13T16:50:35.087606+0000","last_undegraded":"2024-09-13T16:50:35.087606+0000","last_fullsized":"2024-09-13T16:50:35.087606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:36.425638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456657+0000","last_change":"2024-09-13T16:50:30.102739+0000","last_active":"2024-09-13T16:50:35.456657+0000","last_peered":"2024-09-13T16:50:35.456657+0000","last_clean":"2024-09-13T16:50:35.456657+0000","last_became_active":"2024-09-13T16:50:30.102448+0000","last_became_peered":"2024-09-13T16:50:30.102448+0000","last_unstale":"2024-09-13T16:50:35.456657+0000","last_undegraded":"2024-09-13T16:50:35.456657+0000","last_fullsized":"2024-09-13T16:50:35.456657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:38.882189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501017+0000","last_change":"2024-09-13T16:50:30.597447+0000","last_active":"2024-09-13T16:50:35.501017+0000","last_peered":"2024-09-13T16:50:35.501017+0000","last_clean":"2024-09-13T16:50:35.501017+0000","last_became_active":"2024-09-13T16:50:30.592891+0000","last_became_peered":"2024-09-13T16:50:30.592891+0000","last_unstale":"2024-09-13T16:50:35.501017+0000","last_undegraded":"2024-09-13T16:50:35.501017+0000","last_fullsized":"2024-09-13T16:50:35.501017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:13.892340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456015+0000","last_change":"2024-09-13T16:50:34.630131+0000","last_active":"2024-09-13T16:50:35.456015+0000","last_peered":"2024-09-13T16:50:35.456015+0000","last_clean":"2024-09-13T16:50:35.456015+0000","last_became_active":"2024-09-13T16:50:34.629601+0000","last_became_peered":"2024-09-13T16:50:34.629601+0000","last_unstale":"2024-09-13T16:50:35.456015+0000","last_undegraded":"2024-09-13T16:50:35.456015+0000","last_fullsized":"2024-09-13T16:50:35.456015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:38.058431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.090962+0000","last_change":"2024-09-13T16:50:30.612982+0000","last_active":"2024-09-13T16:50:35.090962+0000","last_peered":"2024-09-13T16:50:35.090962+0000","last_clean":"2024-09-13T16:50:35.090962+0000","last_became_active":"2024-09-13T16:50:30.612577+0000","last_became_peered":"2024-09-13T16:50:30.612577+0000","last_unstale":"2024-09-13T16:50:35.090962+0000","last_undegraded":"2024-09-13T16:50:35.090962+0000","last_fullsized":"2024-09-13T16:50:35.090962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087762+0000","last_change":"2024-09-13T16:50:30.594530+0000","last_active":"2024-09-13T16:50:35.087762+0000","last_peered":"2024-09-13T16:50:35.087762+0000","last_clean":"2024-09-13T16:50:35.087762+0000","last_became_active":"2024-09-13T16:50:30.594248+0000","last_became_peered":"2024-09-13T16:50:30.594248+0000","last_unstale":"2024-09-13T16:50:35.087762+0000","last_undegraded":"2024-09-13T16:50:35.087762+0000","last_fullsized":"2024-09-13T16:50:35.087762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:22.950313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091814+0000","last_change":"2024-09-13T16:50:30.602412+0000","last_active":"2024-09-13T16:50:35.091814+0000","last_peered":"2024-09-13T16:50:35.091814+0000","last_clean":"2024-09-13T16:50:35.091814+0000","last_became_active":"2024-09-13T16:50:30.602146+0000","last_became_peered":"2024-09-13T16:50:30.602146+0000","last_unstale":"2024-09-13T16:50:35.091814+0000","last_undegraded":"2024-09-13T16:50:35.091814+0000","last_fullsized":"2024-09-13T16:50:35.091814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:35.586398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462908+0000","last_change":"2024-09-13T16:50:30.606003+0000","last_active":"2024-09-13T16:50:35.462908+0000","last_peered":"2024-09-13T16:50:35.462908+0000","last_clean":"2024-09-13T16:50:35.462908+0000","last_became_active":"2024-09-13T16:50:30.605562+0000","last_became_peered":"2024-09-13T16:50:30.605562+0000","last_unstale":"2024-09-13T16:50:35.462908+0000","last_undegraded":"2024-09-13T16:50:35.462908+0000","last_fullsized":"2024-09-13T16:50:35.462908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456431+0000","last_change":"2024-09-13T16:50:30.617701+0000","last_active":"2024-09-13T16:50:35.456431+0000","last_peered":"2024-09-13T16:50:35.456431+0000","last_clean":"2024-09-13T16:50:35.456431+0000","last_became_active":"2024-09-13T16:50:30.616581+0000","last_became_peered":"2024-09-13T16:50:30.616581+0000","last_unstale":"2024-09-13T16:50:35.456431+0000","last_undegraded":"2024-09-13T16:50:35.456431+0000","last_fullsized":"2024-09-13T16:50:35.456431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:52.041529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401854+0000","last_change":"2024-09-13T16:50:30.599260+0000","last_active":"2024-09-13T16:50:35.401854+0000","last_peered":"2024-09-13T16:50:35.401854+0000","last_clean":"2024-09-13T16:50:35.401854+0000","last_became_active":"2024-09-13T16:50:30.599018+0000","last_became_peered":"2024-09-13T16:50:30.599018+0000","last_unstale":"2024-09-13T16:50:35.401854+0000","last_undegraded":"2024-09-13T16:50:35.401854+0000","last_fullsized":"2024-09-13T16:50:35.401854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093088+0000","last_change":"2024-09-13T16:50:30.613429+0000","last_active":"2024-09-13T16:50:35.093088+0000","last_peered":"2024-09-13T16:50:35.093088+0000","last_clean":"2024-09-13T16:50:35.093088+0000","last_became_active":"2024-09-13T16:50:30.613163+0000","last_became_peered":"2024-09-13T16:50:30.613163+0000","last_unstale":"2024-09-13T16:50:35.093088+0000","last_undegraded":"2024-09-13T16:50:35.093088+0000","last_fullsized":"2024-09-13T16:50:35.093088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456736+0000","last_change":"2024-09-13T16:50:30.607903+0000","last_active":"2024-09-13T16:50:35.456736+0000","last_peered":"2024-09-13T16:50:35.456736+0000","last_clean":"2024-09-13T16:50:35.456736+0000","last_became_active":"2024-09-13T16:50:30.607618+0000","last_became_peered":"2024-09-13T16:50:30.607618+0000","last_unstale":"2024-09-13T16:50:35.456736+0000","last_undegraded":"2024-09-13T16:50:35.456736+0000","last_fullsized":"2024-09-13T16:50:35.456736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091481+0000","last_change":"2024-09-13T16:50:30.603058+0000","last_active":"2024-09-13T16:50:35.091481+0000","last_peered":"2024-09-13T16:50:35.091481+0000","last_clean":"2024-09-13T16:50:35.091481+0000","last_became_active":"2024-09-13T16:50:30.602556+0000","last_became_peered":"2024-09-13T16:50:30.602556+0000","last_unstale":"2024-09-13T16:50:35.091481+0000","last_undegraded":"2024-09-13T16:50:35.091481+0000","last_fullsized":"2024-09-13T16:50:35.091481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456543+0000","last_change":"2024-09-13T16:50:30.599518+0000","last_active":"2024-09-13T16:50:35.456543+0000","last_peered":"2024-09-13T16:50:35.456543+0000","last_clean":"2024-09-13T16:50:35.456543+0000","last_became_active":"2024-09-13T16:50:30.599242+0000","last_became_peered":"2024-09-13T16:50:30.599242+0000","last_unstale":"2024-09-13T16:50:35.456543+0000","last_undegraded":"2024-09-13T16:50:35.456543+0000","last_fullsized":"2024-09-13T16:50:35.456543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091768+0000","last_change":"2024-09-13T16:50:30.610239+0000","last_active":"2024-09-13T16:50:35.091768+0000","last_peered":"2024-09-13T16:50:35.091768+0000","last_clean":"2024-09-13T16:50:35.091768+0000","last_became_active":"2024-09-13T16:50:30.609978+0000","last_became_peered":"2024-09-13T16:50:30.609978+0000","last_unstale":"2024-09-13T16:50:35.091768+0000","last_undegraded":"2024-09-13T16:50:35.091768+0000","last_fullsized":"2024-09-13T16:50:35.091768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:28.715573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462797+0000","last_change":"2024-09-13T16:50:30.618976+0000","last_active":"2024-09-13T16:50:35.462797+0000","last_peered":"2024-09-13T16:50:35.462797+0000","last_clean":"2024-09-13T16:50:35.462797+0000","last_became_active":"2024-09-13T16:50:30.618644+0000","last_became_peered":"2024-09-13T16:50:30.618644+0000","last_unstale":"2024-09-13T16:50:35.462797+0000","last_undegraded":"2024-09-13T16:50:35.462797+0000","last_fullsized":"2024-09-13T16:50:35.462797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:52.041529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092551+0000","last_change":"2024-09-13T16:50:30.609748+0000","last_active":"2024-09-13T16:50:35.092551+0000","last_peered":"2024-09-13T16:50:35.092551+0000","last_clean":"2024-09-13T16:50:35.092551+0000","last_became_active":"2024-09-13T16:50:30.609504+0000","last_became_peered":"2024-09-13T16:50:30.609504+0000","last_unstale":"2024-09-13T16:50:35.092551+0000","last_undegraded":"2024-09-13T16:50:35.092551+0000","last_fullsized":"2024-09-13T16:50:35.092551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500181+0000","last_change":"2024-09-13T16:50:30.618417+0000","last_active":"2024-09-13T16:50:35.500181+0000","last_peered":"2024-09-13T16:50:35.500181+0000","last_clean":"2024-09-13T16:50:35.500181+0000","last_became_active":"2024-09-13T16:50:30.618114+0000","last_became_peered":"2024-09-13T16:50:30.618114+0000","last_unstale":"2024-09-13T16:50:35.500181+0000","last_undegraded":"2024-09-13T16:50:35.500181+0000","last_fullsized":"2024-09-13T16:50:35.500181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:25.758134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.088098+0000","last_change":"2024-09-13T16:50:30.604041+0000","last_active":"2024-09-13T16:50:35.088098+0000","last_peered":"2024-09-13T16:50:35.088098+0000","last_clean":"2024-09-13T16:50:35.088098+0000","last_became_active":"2024-09-13T16:50:30.599336+0000","last_became_peered":"2024-09-13T16:50:30.599336+0000","last_unstale":"2024-09-13T16:50:35.088098+0000","last_undegraded":"2024-09-13T16:50:35.088098+0000","last_fullsized":"2024-09-13T16:50:35.088098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:13.892340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093370+0000","last_change":"2024-09-13T16:50:30.610847+0000","last_active":"2024-09-13T16:50:35.093370+0000","last_peered":"2024-09-13T16:50:35.093370+0000","last_clean":"2024-09-13T16:50:35.093370+0000","last_became_active":"2024-09-13T16:50:30.610581+0000","last_became_peered":"2024-09-13T16:50:30.610581+0000","last_unstale":"2024-09-13T16:50:35.093370+0000","last_undegraded":"2024-09-13T16:50:35.093370+0000","last_fullsized":"2024-09-13T16:50:35.093370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:21.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091388+0000","last_change":"2024-09-13T16:50:30.618144+0000","last_active":"2024-09-13T16:50:35.091388+0000","last_peered":"2024-09-13T16:50:35.091388+0000","last_clean":"2024-09-13T16:50:35.091388+0000","last_became_active":"2024-09-13T16:50:30.617861+0000","last_became_peered":"2024-09-13T16:50:30.617861+0000","last_unstale":"2024-09-13T16:50:35.091388+0000","last_undegraded":"2024-09-13T16:50:35.091388+0000","last_fullsized":"2024-09-13T16:50:35.091388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.400884+0000","last_change":"2024-09-13T16:50:30.590573+0000","last_active":"2024-09-13T16:50:35.400884+0000","last_peered":"2024-09-13T16:50:35.400884+0000","last_clean":"2024-09-13T16:50:35.400884+0000","last_became_active":"2024-09-13T16:50:30.590338+0000","last_became_peered":"2024-09-13T16:50:30.590338+0000","last_unstale":"2024-09-13T16:50:35.400884+0000","last_undegraded":"2024-09-13T16:50:35.400884+0000","last_fullsized":"2024-09-13T16:50:35.400884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462638+0000","last_change":"2024-09-13T16:50:30.615260+0000","last_active":"2024-09-13T16:50:35.462638+0000","last_peered":"2024-09-13T16:50:35.462638+0000","last_clean":"2024-09-13T16:50:35.462638+0000","last_became_active":"2024-09-13T16:50:30.614529+0000","last_became_peered":"2024-09-13T16:50:30.614529+0000","last_unstale":"2024-09-13T16:50:35.462638+0000","last_undegraded":"2024-09-13T16:50:35.462638+0000","last_fullsized":"2024-09-13T16:50:35.462638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.455914+0000","last_change":"2024-09-13T16:50:30.603994+0000","last_active":"2024-09-13T16:50:35.455914+0000","last_peered":"2024-09-13T16:50:35.455914+0000","last_clean":"2024-09-13T16:50:35.455914+0000","last_became_active":"2024-09-13T16:50:30.603610+0000","last_became_peered":"2024-09-13T16:50:30.603610+0000","last_unstale":"2024-09-13T16:50:35.455914+0000","last_undegraded":"2024-09-13T16:50:35.455914+0000","last_fullsized":"2024-09-13T16:50:35.455914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.502132+0000","last_change":"2024-09-13T16:50:30.603873+0000","last_active":"2024-09-13T16:50:35.502132+0000","last_peered":"2024-09-13T16:50:35.502132+0000","last_clean":"2024-09-13T16:50:35.502132+0000","last_became_active":"2024-09-13T16:50:30.603571+0000","last_became_peered":"2024-09-13T16:50:30.603571+0000","last_unstale":"2024-09-13T16:50:35.502132+0000","last_undegraded":"2024-09-13T16:50:35.502132+0000","last_fullsized":"2024-09-13T16:50:35.502132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:22.950313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092808+0000","last_change":"2024-09-13T16:50:30.607038+0000","last_active":"2024-09-13T16:50:35.092808+0000","last_peered":"2024-09-13T16:50:35.092808+0000","last_clean":"2024-09-13T16:50:35.092808+0000","last_became_active":"2024-09-13T16:50:30.606788+0000","last_became_peered":"2024-09-13T16:50:30.606788+0000","last_unstale":"2024-09-13T16:50:35.092808+0000","last_undegraded":"2024-09-13T16:50:35.092808+0000","last_fullsized":"2024-09-13T16:50:35.092808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093030+0000","last_change":"2024-09-13T16:50:30.599427+0000","last_active":"2024-09-13T16:50:35.093030+0000","last_peered":"2024-09-13T16:50:35.093030+0000","last_clean":"2024-09-13T16:50:35.093030+0000","last_became_active":"2024-09-13T16:50:30.599149+0000","last_became_peered":"2024-09-13T16:50:30.599149+0000","last_unstale":"2024-09-13T16:50:35.093030+0000","last_undegraded":"2024-09-13T16:50:35.093030+0000","last_fullsized":"2024-09-13T16:50:35.093030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500732+0000","last_change":"2024-09-13T16:50:30.615710+0000","last_active":"2024-09-13T16:50:35.500732+0000","last_peered":"2024-09-13T16:50:35.500732+0000","last_clean":"2024-09-13T16:50:35.500732+0000","last_became_active":"2024-09-13T16:50:30.615444+0000","last_became_peered":"2024-09-13T16:50:30.615444+0000","last_unstale":"2024-09-13T16:50:35.500732+0000","last_undegraded":"2024-09-13T16:50:35.500732+0000","last_fullsized":"2024-09-13T16:50:35.500732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:35.586398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456590+0000","last_change":"2024-09-13T16:50:30.599700+0000","last_active":"2024-09-13T16:50:35.456590+0000","last_peered":"2024-09-13T16:50:35.456590+0000","last_clean":"2024-09-13T16:50:35.456590+0000","last_became_active":"2024-09-13T16:50:30.599132+0000","last_became_peered":"2024-09-13T16:50:30.599132+0000","last_unstale":"2024-09-13T16:50:35.456590+0000","last_undegraded":"2024-09-13T16:50:35.456590+0000","last_fullsized":"2024-09-13T16:50:35.456590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498763+0000","last_change":"2024-09-13T16:50:30.617544+0000","last_active":"2024-09-13T16:50:35.498763+0000","last_peered":"2024-09-13T16:50:35.498763+0000","last_clean":"2024-09-13T16:50:35.498763+0000","last_became_active":"2024-09-13T16:50:30.617008+0000","last_became_peered":"2024-09-13T16:50:30.617008+0000","last_unstale":"2024-09-13T16:50:35.498763+0000","last_undegraded":"2024-09-13T16:50:35.498763+0000","last_fullsized":"2024-09-13T16:50:35.498763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091720+0000","last_change":"2024-09-13T16:50:30.613899+0000","last_active":"2024-09-13T16:50:35.091720+0000","last_peered":"2024-09-13T16:50:35.091720+0000","last_clean":"2024-09-13T16:50:35.091720+0000","last_became_active":"2024-09-13T16:50:30.613520+0000","last_became_peered":"2024-09-13T16:50:30.613520+0000","last_unstale":"2024-09-13T16:50:35.091720+0000","last_undegraded":"2024-09-13T16:50:35.091720+0000","last_fullsized":"2024-09-13T16:50:35.091720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498780+0000","last_change":"2024-09-13T16:50:30.617169+0000","last_active":"2024-09-13T16:50:35.498780+0000","last_peered":"2024-09-13T16:50:35.498780+0000","last_clean":"2024-09-13T16:50:35.498780+0000","last_became_active":"2024-09-13T16:50:30.616608+0000","last_became_peered":"2024-09-13T16:50:30.616608+0000","last_unstale":"2024-09-13T16:50:35.498780+0000","last_undegraded":"2024-09-13T16:50:35.498780+0000","last_fullsized":"2024-09-13T16:50:35.498780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.117003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092229+0000","last_change":"2024-09-13T16:50:30.601494+0000","last_active":"2024-09-13T16:50:35.092229+0000","last_peered":"2024-09-13T16:50:35.092229+0000","last_clean":"2024-09-13T16:50:35.092229+0000","last_became_active":"2024-09-13T16:50:30.601235+0000","last_became_peered":"2024-09-13T16:50:30.601235+0000","last_unstale":"2024-09-13T16:50:35.092229+0000","last_undegraded":"2024-09-13T16:50:35.092229+0000","last_fullsized":"2024-09-13T16:50:35.092229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:54.244738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463054+0000","last_change":"2024-09-13T16:50:30.611789+0000","last_active":"2024-09-13T16:50:35.463054+0000","last_peered":"2024-09-13T16:50:35.463054+0000","last_clean":"2024-09-13T16:50:35.463054+0000","last_became_active":"2024-09-13T16:50:30.608646+0000","last_became_peered":"2024-09-13T16:50:30.608646+0000","last_unstale":"2024-09-13T16:50:35.463054+0000","last_undegraded":"2024-09-13T16:50:35.463054+0000","last_fullsized":"2024-09-13T16:50:35.463054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.145790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402335+0000","last_change":"2024-09-13T16:50:30.609892+0000","last_active":"2024-09-13T16:50:35.402335+0000","last_peered":"2024-09-13T16:50:35.402335+0000","last_clean":"2024-09-13T16:50:35.402335+0000","last_became_active":"2024-09-13T16:50:30.609664+0000","last_became_peered":"2024-09-13T16:50:30.609664+0000","last_unstale":"2024-09-13T16:50:35.402335+0000","last_undegraded":"2024-09-13T16:50:35.402335+0000","last_fullsized":"2024-09-13T16:50:35.402335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500628+0000","last_change":"2024-09-13T16:50:30.607977+0000","last_active":"2024-09-13T16:50:35.500628+0000","last_peered":"2024-09-13T16:50:35.500628+0000","last_clean":"2024-09-13T16:50:35.500628+0000","last_became_active":"2024-09-13T16:50:30.607735+0000","last_became_peered":"2024-09-13T16:50:30.607735+0000","last_unstale":"2024-09-13T16:50:35.500628+0000","last_undegraded":"2024-09-13T16:50:35.500628+0000","last_fullsized":"2024-09-13T16:50:35.500628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:21.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":864256,"data_stored":697795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":278528,"data_stored":106971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.839}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.913}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8979999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":106845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9139999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":697851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:50:57.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:50:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.281+0000 7f6d6f200700 1 Processor -- start 2024-09-13T16:50:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.281+0000 7f6d6f200700 1 -- start start 2024-09-13T16:50:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.281+0000 7f6d6f200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d681443b0 0x7f6d680b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6d6806ccf0 0x7f6d6815b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d680ce8e0 con 0x7f6d68144790 2024-09-13T16:50:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d680ccd30 con 0x7f6d681443b0 2024-09-13T16:50:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d680cd010 con 0x7f6d6806ccf0 2024-09-13T16:50:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6de00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d681443b0 0x7f6d680b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6de00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d681443b0 0x7f6d680b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54386/0 (socket says 172.21.15.3:54386) 2024-09-13T16:50:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6de00700 1 -- 172.21.15.3:0/1081665742 learned_addr learned my addr 172.21.15.3:0/1081665742 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6e800700 1 --2- 172.21.15.3:0/1081665742 >> v2:172.21.15.157:3300/0 conn(0x7f6d6806ccf0 0x7f6d6815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6de00700 1 -- 172.21.15.3:0/1081665742 >> v2:172.21.15.157:3300/0 conn(0x7f6d6806ccf0 msgr2=0x7f6d6815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6de00700 1 --2- 172.21.15.3:0/1081665742 >> v2:172.21.15.157:3300/0 conn(0x7f6d6806ccf0 0x7f6d6815b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6de00700 1 -- 172.21.15.3:0/1081665742 >> v1:172.21.15.157:6789/0 conn(0x7f6d68144790 legacy=0x7f6d6806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6de00700 1 -- 172.21.15.3:0/1081665742 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d680cd2f0 con 0x7f6d681443b0 2024-09-13T16:50:57.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6de00700 1 --2- 172.21.15.3:0/1081665742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 0x7f6d680b39f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d680b3280 tx=0x7f6d5c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec60455f17576f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6ca00700 1 -- 172.21.15.3:0/1081665742 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c004230 con 0x7f6d681443b0 2024-09-13T16:50:57.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6ca00700 1 -- 172.21.15.3:0/1081665742 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c004390 con 0x7f6d681443b0 2024-09-13T16:50:57.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6ca00700 1 -- 172.21.15.3:0/1081665742 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0036c0 con 0x7f6d681443b0 2024-09-13T16:50:57.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 -- 172.21.15.3:0/1081665742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 msgr2=0x7f6d680b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1081665742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 0x7f6d680b39f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d680b3280 tx=0x7f6d5c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 -- 172.21.15.3:0/1081665742 shutdown_connections 2024-09-13T16:50:57.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1081665742 >> v2:172.21.15.157:3300/0 conn(0x7f6d6806ccf0 0x7f6d6815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1081665742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 0x7f6d680b39f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 -- 172.21.15.3:0/1081665742 >> 172.21.15.3:0/1081665742 conn(0x7f6d680b0cf0 msgr2=0x7f6d680b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:57.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 -- 172.21.15.3:0/1081665742 shutdown_connections 2024-09-13T16:50:57.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.285+0000 7f6d6f200700 1 -- 172.21.15.3:0/1081665742 wait complete. 2024-09-13T16:50:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6f200700 1 Processor -- start 2024-09-13T16:50:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6f200700 1 -- start start 2024-09-13T16:50:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d6806ccf0 0x7f6d680ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6f200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 0x7f6d680acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6f200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d680ad100 0x7f6d6813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6f200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680ae1d0 con 0x7f6d681443b0 2024-09-13T16:50:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680ab740 con 0x7f6d6806ccf0 2024-09-13T16:50:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6f200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680ad5c0 con 0x7f6d680ad100 2024-09-13T16:50:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d6806ccf0 0x7f6d680ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 0x7f6d680acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d6806ccf0 0x7f6d680ab200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:51740/0 (socket says 172.21.15.3:51740) 2024-09-13T16:50:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.289+0000 7f6d6de00700 1 -- 172.21.15.3:0/1531613418 learned_addr learned my addr 172.21.15.3:0/1531613418 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6e800700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d680ad100 0x7f6d6813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6de00700 1 -- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d680ad100 msgr2=0x7f6d6813ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6de00700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d680ad100 0x7f6d6813ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6de00700 1 -- 172.21.15.3:0/1531613418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 msgr2=0x7f6d680acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6de00700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 0x7f6d680acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6de00700 1 -- 172.21.15.3:0/1531613418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c009570 con 0x7f6d6806ccf0 2024-09-13T16:50:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6d400700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 0x7f6d680acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6de00700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d6806ccf0 0x7f6d680ab200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c004000 tx=0x7f6d5c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d62000700 1 -- 172.21.15.3:0/1531613418 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00b660 con 0x7f6d6806ccf0 2024-09-13T16:50:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d62000700 1 -- 172.21.15.3:0/1531613418 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00b7c0 con 0x7f6d6806ccf0 2024-09-13T16:50:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d680ad8a0 con 0x7f6d6806ccf0 2024-09-13T16:50:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d62000700 1 -- 172.21.15.3:0/1531613418 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00bb70 con 0x7f6d6806ccf0 2024-09-13T16:50:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d680ade60 con 0x7f6d6806ccf0 2024-09-13T16:50:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.293+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d680b39f0 con 0x7f6d6806ccf0 2024-09-13T16:50:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.297+0000 7f6d62000700 1 -- 172.21.15.3:0/1531613418 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d5c00bcd0 con 0x7f6d6806ccf0 2024-09-13T16:50:57.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.297+0000 7f6d62000700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d4c066780 0x7f6d4c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.297+0000 7f6d62000700 1 -- 172.21.15.3:0/1531613418 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6d5c085f50 con 0x7f6d6806ccf0 2024-09-13T16:50:57.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.297+0000 7f6d6d400700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d4c066780 0x7f6d4c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.301+0000 7f6d6d400700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d4c066780 0x7f6d4c068c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d6813bcd0 tx=0x7f6d58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.305+0000 7f6d62000700 1 -- 172.21.15.3:0/1531613418 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6d5c053040 con 0x7f6d6806ccf0 2024-09-13T16:50:57.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.497+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d6806bce0 con 0x7f6d4c066780 2024-09-13T16:50:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.513+0000 7f6d62000700 1 -- 172.21.15.3:0/1531613418 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345055 (secure 0 0 0) 0x7f6d6806bce0 con 0x7f6d4c066780 2024-09-13T16:50:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d4c066780 msgr2=0x7f6d4c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d4c066780 0x7f6d4c068c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d6813bcd0 tx=0x7f6d58009040 comp rx=0 tx=0).stop 2024-09-13T16:50:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d6806ccf0 msgr2=0x7f6d680ab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d6806ccf0 0x7f6d680ab200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c004000 tx=0x7f6d5c004770 comp rx=0 tx=0).stop 2024-09-13T16:50:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 shutdown_connections 2024-09-13T16:50:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d4c066780 0x7f6d4c068c40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d680ad100 0x7f6d6813ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d681443b0 0x7f6d680acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 --2- 172.21.15.3:0/1531613418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d6806ccf0 0x7f6d680ab200 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 >> 172.21.15.3:0/1531613418 conn(0x7f6d680b0cf0 msgr2=0x7f6d681424b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:57.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 shutdown_connections 2024-09-13T16:50:57.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.525+0000 7f6d6f200700 1 -- 172.21.15.3:0/1531613418 wait complete. 2024-09-13T16:50:57.533 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:50:57.554 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-13T16:50:57.201893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227664,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754747056,"statfs":{"total":773094113280,"available":772860985344,"internally_reserved":0,"allocated":3383296,"data_stored":2044682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005576"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463496+0000","last_change":"2024-09-13T16:50:30.106124+0000","last_active":"2024-09-13T16:50:35.463496+0000","last_peered":"2024-09-13T16:50:35.463496+0000","last_clean":"2024-09-13T16:50:35.463496+0000","last_became_active":"2024-09-13T16:50:30.105874+0000","last_became_peered":"2024-09-13T16:50:30.105874+0000","last_unstale":"2024-09-13T16:50:35.463496+0000","last_undegraded":"2024-09-13T16:50:35.463496+0000","last_fullsized":"2024-09-13T16:50:35.463496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:38.882189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456272+0000","last_change":"2024-09-13T16:50:30.598817+0000","last_active":"2024-09-13T16:50:35.456272+0000","last_peered":"2024-09-13T16:50:35.456272+0000","last_clean":"2024-09-13T16:50:35.456272+0000","last_became_active":"2024-09-13T16:50:30.598483+0000","last_became_peered":"2024-09-13T16:50:30.598483+0000","last_unstale":"2024-09-13T16:50:35.456272+0000","last_undegraded":"2024-09-13T16:50:35.456272+0000","last_fullsized":"2024-09-13T16:50:35.456272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.511304+0000","last_change":"2024-09-13T16:50:35.507978+0000","last_active":"2024-09-13T16:50:35.511304+0000","last_peered":"2024-09-13T16:50:35.511304+0000","last_clean":"2024-09-13T16:50:35.511304+0000","last_became_active":"2024-09-13T16:50:35.506655+0000","last_became_peered":"2024-09-13T16:50:35.506655+0000","last_unstale":"2024-09-13T16:50:35.511304+0000","last_undegraded":"2024-09-13T16:50:35.511304+0000","last_fullsized":"2024-09-13T16:50:35.511304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:29.062934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456822+0000","last_change":"2024-09-13T16:50:30.101330+0000","last_active":"2024-09-13T16:50:35.456822+0000","last_peered":"2024-09-13T16:50:35.456822+0000","last_clean":"2024-09-13T16:50:35.456822+0000","last_became_active":"2024-09-13T16:50:30.101038+0000","last_became_peered":"2024-09-13T16:50:30.101038+0000","last_unstale":"2024-09-13T16:50:35.456822+0000","last_undegraded":"2024-09-13T16:50:35.456822+0000","last_fullsized":"2024-09-13T16:50:35.456822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463263+0000","last_change":"2024-09-13T16:50:30.609337+0000","last_active":"2024-09-13T16:50:35.463263+0000","last_peered":"2024-09-13T16:50:35.463263+0000","last_clean":"2024-09-13T16:50:35.463263+0000","last_became_active":"2024-09-13T16:50:30.609098+0000","last_became_peered":"2024-09-13T16:50:30.609098+0000","last_unstale":"2024-09-13T16:50:35.463263+0000","last_undegraded":"2024-09-13T16:50:35.463263+0000","last_fullsized":"2024-09-13T16:50:35.463263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498079+0000","last_change":"2024-09-13T16:50:34.633734+0000","last_active":"2024-09-13T16:50:35.498079+0000","last_peered":"2024-09-13T16:50:35.498079+0000","last_clean":"2024-09-13T16:50:35.498079+0000","last_became_active":"2024-09-13T16:50:34.632931+0000","last_became_peered":"2024-09-13T16:50:34.632931+0000","last_unstale":"2024-09-13T16:50:35.498079+0000","last_undegraded":"2024-09-13T16:50:35.498079+0000","last_fullsized":"2024-09-13T16:50:35.498079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:37.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500017+0000","last_change":"2024-09-13T16:50:30.108543+0000","last_active":"2024-09-13T16:50:35.500017+0000","last_peered":"2024-09-13T16:50:35.500017+0000","last_clean":"2024-09-13T16:50:35.500017+0000","last_became_active":"2024-09-13T16:50:30.108282+0000","last_became_peered":"2024-09-13T16:50:30.108282+0000","last_unstale":"2024-09-13T16:50:35.500017+0000","last_undegraded":"2024-09-13T16:50:35.500017+0000","last_fullsized":"2024-09-13T16:50:35.500017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092867+0000","last_change":"2024-09-13T16:50:30.594489+0000","last_active":"2024-09-13T16:50:35.092867+0000","last_peered":"2024-09-13T16:50:35.092867+0000","last_clean":"2024-09-13T16:50:35.092867+0000","last_became_active":"2024-09-13T16:50:30.593342+0000","last_became_peered":"2024-09-13T16:50:30.593342+0000","last_unstale":"2024-09-13T16:50:35.092867+0000","last_undegraded":"2024-09-13T16:50:35.092867+0000","last_fullsized":"2024-09-13T16:50:35.092867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500758+0000","last_change":"2024-09-13T16:50:34.633900+0000","last_active":"2024-09-13T16:50:35.500758+0000","last_peered":"2024-09-13T16:50:35.500758+0000","last_clean":"2024-09-13T16:50:35.500758+0000","last_became_active":"2024-09-13T16:50:34.633428+0000","last_became_peered":"2024-09-13T16:50:34.633428+0000","last_unstale":"2024-09-13T16:50:35.500758+0000","last_undegraded":"2024-09-13T16:50:35.500758+0000","last_fullsized":"2024-09-13T16:50:35.500758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:01.459147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499214+0000","last_change":"2024-09-13T16:50:30.119534+0000","last_active":"2024-09-13T16:50:35.499214+0000","last_peered":"2024-09-13T16:50:35.499214+0000","last_clean":"2024-09-13T16:50:35.499214+0000","last_became_active":"2024-09-13T16:50:30.119177+0000","last_became_peered":"2024-09-13T16:50:30.119177+0000","last_unstale":"2024-09-13T16:50:35.499214+0000","last_undegraded":"2024-09-13T16:50:35.499214+0000","last_fullsized":"2024-09-13T16:50:35.499214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092738+0000","last_change":"2024-09-13T16:50:30.604539+0000","last_active":"2024-09-13T16:50:35.092738+0000","last_peered":"2024-09-13T16:50:35.092738+0000","last_clean":"2024-09-13T16:50:35.092738+0000","last_became_active":"2024-09-13T16:50:30.604192+0000","last_became_peered":"2024-09-13T16:50:30.604192+0000","last_unstale":"2024-09-13T16:50:35.092738+0000","last_undegraded":"2024-09-13T16:50:35.092738+0000","last_fullsized":"2024-09-13T16:50:35.092738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500774+0000","last_change":"2024-09-13T16:50:34.632525+0000","last_active":"2024-09-13T16:50:35.500774+0000","last_peered":"2024-09-13T16:50:35.500774+0000","last_clean":"2024-09-13T16:50:35.500774+0000","last_became_active":"2024-09-13T16:50:34.632114+0000","last_became_peered":"2024-09-13T16:50:34.632114+0000","last_unstale":"2024-09-13T16:50:35.500774+0000","last_undegraded":"2024-09-13T16:50:35.500774+0000","last_fullsized":"2024-09-13T16:50:35.500774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091628+0000","last_change":"2024-09-13T16:50:30.611124+0000","last_active":"2024-09-13T16:50:35.091628+0000","last_peered":"2024-09-13T16:50:35.091628+0000","last_clean":"2024-09-13T16:50:35.091628+0000","last_became_active":"2024-09-13T16:50:30.607850+0000","last_became_peered":"2024-09-13T16:50:30.607850+0000","last_unstale":"2024-09-13T16:50:35.091628+0000","last_undegraded":"2024-09-13T16:50:35.091628+0000","last_fullsized":"2024-09-13T16:50:35.091628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499169+0000","last_change":"2024-09-13T16:50:30.106841+0000","last_active":"2024-09-13T16:50:35.499169+0000","last_peered":"2024-09-13T16:50:35.499169+0000","last_clean":"2024-09-13T16:50:35.499169+0000","last_became_active":"2024-09-13T16:50:30.106581+0000","last_became_peered":"2024-09-13T16:50:30.106581+0000","last_unstale":"2024-09-13T16:50:35.499169+0000","last_undegraded":"2024-09-13T16:50:35.499169+0000","last_fullsized":"2024-09-13T16:50:35.499169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501399+0000","last_change":"2024-09-13T16:50:34.633477+0000","last_active":"2024-09-13T16:50:35.501399+0000","last_peered":"2024-09-13T16:50:35.501399+0000","last_clean":"2024-09-13T16:50:35.501399+0000","last_became_active":"2024-09-13T16:50:34.633072+0000","last_became_peered":"2024-09-13T16:50:34.633072+0000","last_unstale":"2024-09-13T16:50:35.501399+0000","last_undegraded":"2024-09-13T16:50:35.501399+0000","last_fullsized":"2024-09-13T16:50:35.501399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:14.360209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456436+0000","last_change":"2024-09-13T16:50:30.611139+0000","last_active":"2024-09-13T16:50:35.456436+0000","last_peered":"2024-09-13T16:50:35.456436+0000","last_clean":"2024-09-13T16:50:35.456436+0000","last_became_active":"2024-09-13T16:50:30.610455+0000","last_became_peered":"2024-09-13T16:50:30.610455+0000","last_unstale":"2024-09-13T16:50:35.456436+0000","last_undegraded":"2024-09-13T16:50:35.456436+0000","last_fullsized":"2024-09-13T16:50:35.456436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402349+0000","last_change":"2024-09-13T16:50:30.113719+0000","last_active":"2024-09-13T16:50:35.402349+0000","last_peered":"2024-09-13T16:50:35.402349+0000","last_clean":"2024-09-13T16:50:35.402349+0000","last_became_active":"2024-09-13T16:50:30.111437+0000","last_became_peered":"2024-09-13T16:50:30.111437+0000","last_unstale":"2024-09-13T16:50:35.402349+0000","last_undegraded":"2024-09-13T16:50:35.402349+0000","last_fullsized":"2024-09-13T16:50:35.402349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501838+0000","last_change":"2024-09-13T16:50:34.638895+0000","last_active":"2024-09-13T16:50:35.501838+0000","last_peered":"2024-09-13T16:50:35.501838+0000","last_clean":"2024-09-13T16:50:35.501838+0000","last_became_active":"2024-09-13T16:50:34.638431+0000","last_became_peered":"2024-09-13T16:50:34.638431+0000","last_unstale":"2024-09-13T16:50:35.501838+0000","last_undegraded":"2024-09-13T16:50:35.501838+0000","last_fullsized":"2024-09-13T16:50:35.501838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:51.705201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.457086+0000","last_change":"2024-09-13T16:50:30.599950+0000","last_active":"2024-09-13T16:50:35.457086+0000","last_peered":"2024-09-13T16:50:35.457086+0000","last_clean":"2024-09-13T16:50:35.457086+0000","last_became_active":"2024-09-13T16:50:30.599401+0000","last_became_peered":"2024-09-13T16:50:30.599401+0000","last_unstale":"2024-09-13T16:50:35.457086+0000","last_undegraded":"2024-09-13T16:50:35.457086+0000","last_fullsized":"2024-09-13T16:50:35.457086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401297+0000","last_change":"2024-09-13T16:50:30.110698+0000","last_active":"2024-09-13T16:50:35.401297+0000","last_peered":"2024-09-13T16:50:35.401297+0000","last_clean":"2024-09-13T16:50:35.401297+0000","last_became_active":"2024-09-13T16:50:30.110343+0000","last_became_peered":"2024-09-13T16:50:30.110343+0000","last_unstale":"2024-09-13T16:50:35.401297+0000","last_undegraded":"2024-09-13T16:50:35.401297+0000","last_fullsized":"2024-09-13T16:50:35.401297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.512080+0000","last_change":"2024-09-13T16:50:35.510605+0000","last_active":"2024-09-13T16:50:35.512080+0000","last_peered":"2024-09-13T16:50:35.512080+0000","last_clean":"2024-09-13T16:50:35.512080+0000","last_became_active":"2024-09-13T16:50:35.510230+0000","last_became_peered":"2024-09-13T16:50:35.510230+0000","last_unstale":"2024-09-13T16:50:35.512080+0000","last_undegraded":"2024-09-13T16:50:35.512080+0000","last_fullsized":"2024-09-13T16:50:35.512080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500916+0000","last_change":"2024-09-13T16:50:30.608538+0000","last_active":"2024-09-13T16:50:35.500916+0000","last_peered":"2024-09-13T16:50:35.500916+0000","last_clean":"2024-09-13T16:50:35.500916+0000","last_became_active":"2024-09-13T16:50:30.608195+0000","last_became_peered":"2024-09-13T16:50:30.608195+0000","last_unstale":"2024-09-13T16:50:35.500916+0000","last_undegraded":"2024-09-13T16:50:35.500916+0000","last_fullsized":"2024-09-13T16:50:35.500916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:48.431406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.497470+0000","last_change":"2024-09-13T16:50:30.110135+0000","last_active":"2024-09-13T16:50:35.497470+0000","last_peered":"2024-09-13T16:50:35.497470+0000","last_clean":"2024-09-13T16:50:35.497470+0000","last_became_active":"2024-09-13T16:50:30.109520+0000","last_became_peered":"2024-09-13T16:50:30.109520+0000","last_unstale":"2024-09-13T16:50:35.497470+0000","last_undegraded":"2024-09-13T16:50:35.497470+0000","last_fullsized":"2024-09-13T16:50:35.497470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501389+0000","last_change":"2024-09-13T16:50:34.635782+0000","last_active":"2024-09-13T16:50:35.501389+0000","last_peered":"2024-09-13T16:50:35.501389+0000","last_clean":"2024-09-13T16:50:35.501389+0000","last_became_active":"2024-09-13T16:50:34.635349+0000","last_became_peered":"2024-09-13T16:50:34.635349+0000","last_unstale":"2024-09-13T16:50:35.501389+0000","last_undegraded":"2024-09-13T16:50:35.501389+0000","last_fullsized":"2024-09-13T16:50:35.501389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:58.754567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463122+0000","last_change":"2024-09-13T16:50:30.614804+0000","last_active":"2024-09-13T16:50:35.463122+0000","last_peered":"2024-09-13T16:50:35.463122+0000","last_clean":"2024-09-13T16:50:35.463122+0000","last_became_active":"2024-09-13T16:50:30.614458+0000","last_became_peered":"2024-09-13T16:50:30.614458+0000","last_unstale":"2024-09-13T16:50:35.463122+0000","last_undegraded":"2024-09-13T16:50:35.463122+0000","last_fullsized":"2024-09-13T16:50:35.463122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093301+0000","last_change":"2024-09-13T16:50:30.107253+0000","last_active":"2024-09-13T16:50:35.093301+0000","last_peered":"2024-09-13T16:50:35.093301+0000","last_clean":"2024-09-13T16:50:35.093301+0000","last_became_active":"2024-09-13T16:50:30.106448+0000","last_became_peered":"2024-09-13T16:50:30.106448+0000","last_unstale":"2024-09-13T16:50:35.093301+0000","last_undegraded":"2024-09-13T16:50:35.093301+0000","last_fullsized":"2024-09-13T16:50:35.093301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.507157+0000","last_change":"2024-09-13T16:50:35.504918+0000","last_active":"2024-09-13T16:50:35.507157+0000","last_peered":"2024-09-13T16:50:35.507157+0000","last_clean":"2024-09-13T16:50:35.507157+0000","last_became_active":"2024-09-13T16:50:35.504252+0000","last_became_peered":"2024-09-13T16:50:35.504252+0000","last_unstale":"2024-09-13T16:50:35.507157+0000","last_undegraded":"2024-09-13T16:50:35.507157+0000","last_fullsized":"2024-09-13T16:50:35.507157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:54.702371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456826+0000","last_change":"2024-09-13T16:50:30.610870+0000","last_active":"2024-09-13T16:50:35.456826+0000","last_peered":"2024-09-13T16:50:35.456826+0000","last_clean":"2024-09-13T16:50:35.456826+0000","last_became_active":"2024-09-13T16:50:30.610473+0000","last_became_peered":"2024-09-13T16:50:30.610473+0000","last_unstale":"2024-09-13T16:50:35.456826+0000","last_undegraded":"2024-09-13T16:50:35.456826+0000","last_fullsized":"2024-09-13T16:50:35.456826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.940724+0000","last_change":"2024-09-13T16:50:30.111250+0000","last_active":"2024-09-13T16:50:35.940724+0000","last_peered":"2024-09-13T16:50:35.940724+0000","last_clean":"2024-09-13T16:50:35.940724+0000","last_became_active":"2024-09-13T16:50:30.109959+0000","last_became_peered":"2024-09-13T16:50:30.109959+0000","last_unstale":"2024-09-13T16:50:35.940724+0000","last_undegraded":"2024-09-13T16:50:35.940724+0000","last_fullsized":"2024-09-13T16:50:35.940724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509887+0000","last_change":"2024-09-13T16:50:35.501491+0000","last_active":"2024-09-13T16:50:35.509887+0000","last_peered":"2024-09-13T16:50:35.509887+0000","last_clean":"2024-09-13T16:50:35.509887+0000","last_became_active":"2024-09-13T16:50:35.500108+0000","last_became_peered":"2024-09-13T16:50:35.500108+0000","last_unstale":"2024-09-13T16:50:35.509887+0000","last_undegraded":"2024-09-13T16:50:35.509887+0000","last_fullsized":"2024-09-13T16:50:35.509887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:30.666178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401804+0000","last_change":"2024-09-13T16:50:30.606888+0000","last_active":"2024-09-13T16:50:35.401804+0000","last_peered":"2024-09-13T16:50:35.401804+0000","last_clean":"2024-09-13T16:50:35.401804+0000","last_became_active":"2024-09-13T16:50:30.605360+0000","last_became_peered":"2024-09-13T16:50:30.605360+0000","last_unstale":"2024-09-13T16:50:35.401804+0000","last_undegraded":"2024-09-13T16:50:35.401804+0000","last_fullsized":"2024-09-13T16:50:35.401804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500383+0000","last_change":"2024-09-13T16:50:30.111119+0000","last_active":"2024-09-13T16:50:35.500383+0000","last_peered":"2024-09-13T16:50:35.500383+0000","last_clean":"2024-09-13T16:50:35.500383+0000","last_became_active":"2024-09-13T16:50:30.110892+0000","last_became_peered":"2024-09-13T16:50:30.110892+0000","last_unstale":"2024-09-13T16:50:35.500383+0000","last_undegraded":"2024-09-13T16:50:35.500383+0000","last_fullsized":"2024-09-13T16:50:35.500383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509685+0000","last_change":"2024-09-13T16:50:35.505149+0000","last_active":"2024-09-13T16:50:35.509685+0000","last_peered":"2024-09-13T16:50:35.509685+0000","last_clean":"2024-09-13T16:50:35.509685+0000","last_became_active":"2024-09-13T16:50:35.502086+0000","last_became_peered":"2024-09-13T16:50:35.502086+0000","last_unstale":"2024-09-13T16:50:35.509685+0000","last_undegraded":"2024-09-13T16:50:35.509685+0000","last_fullsized":"2024-09-13T16:50:35.509685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092835+0000","last_change":"2024-09-13T16:50:30.601700+0000","last_active":"2024-09-13T16:50:35.092835+0000","last_peered":"2024-09-13T16:50:35.092835+0000","last_clean":"2024-09-13T16:50:35.092835+0000","last_became_active":"2024-09-13T16:50:30.601432+0000","last_became_peered":"2024-09-13T16:50:30.601432+0000","last_unstale":"2024-09-13T16:50:35.092835+0000","last_undegraded":"2024-09-13T16:50:35.092835+0000","last_fullsized":"2024-09-13T16:50:35.092835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:25.758134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402007+0000","last_change":"2024-09-13T16:50:30.112636+0000","last_active":"2024-09-13T16:50:35.402007+0000","last_peered":"2024-09-13T16:50:35.402007+0000","last_clean":"2024-09-13T16:50:35.402007+0000","last_became_active":"2024-09-13T16:50:30.112121+0000","last_became_peered":"2024-09-13T16:50:30.112121+0000","last_unstale":"2024-09-13T16:50:35.402007+0000","last_undegraded":"2024-09-13T16:50:35.402007+0000","last_fullsized":"2024-09-13T16:50:35.402007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401192+0000","last_change":"2024-09-13T16:50:34.633860+0000","last_active":"2024-09-13T16:50:35.401192+0000","last_peered":"2024-09-13T16:50:35.401192+0000","last_clean":"2024-09-13T16:50:35.401192+0000","last_became_active":"2024-09-13T16:50:34.633435+0000","last_became_peered":"2024-09-13T16:50:34.633435+0000","last_unstale":"2024-09-13T16:50:35.401192+0000","last_undegraded":"2024-09-13T16:50:35.401192+0000","last_fullsized":"2024-09-13T16:50:35.401192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:27.883765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501232+0000","last_change":"2024-09-13T16:50:30.613267+0000","last_active":"2024-09-13T16:50:35.501232+0000","last_peered":"2024-09-13T16:50:35.501232+0000","last_clean":"2024-09-13T16:50:35.501232+0000","last_became_active":"2024-09-13T16:50:30.612861+0000","last_became_peered":"2024-09-13T16:50:30.612861+0000","last_unstale":"2024-09-13T16:50:35.501232+0000","last_undegraded":"2024-09-13T16:50:35.501232+0000","last_fullsized":"2024-09-13T16:50:35.501232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:27.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091946+0000","last_change":"2024-09-13T16:50:30.110348+0000","last_active":"2024-09-13T16:50:35.091946+0000","last_peered":"2024-09-13T16:50:35.091946+0000","last_clean":"2024-09-13T16:50:35.091946+0000","last_became_active":"2024-09-13T16:50:30.110041+0000","last_became_peered":"2024-09-13T16:50:30.110041+0000","last_unstale":"2024-09-13T16:50:35.091946+0000","last_undegraded":"2024-09-13T16:50:35.091946+0000","last_fullsized":"2024-09-13T16:50:35.091946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401674+0000","last_change":"2024-09-13T16:50:34.633493+0000","last_active":"2024-09-13T16:50:35.401674+0000","last_peered":"2024-09-13T16:50:35.401674+0000","last_clean":"2024-09-13T16:50:35.401674+0000","last_became_active":"2024-09-13T16:50:34.632442+0000","last_became_peered":"2024-09-13T16:50:34.632442+0000","last_unstale":"2024-09-13T16:50:35.401674+0000","last_undegraded":"2024-09-13T16:50:35.401674+0000","last_fullsized":"2024-09-13T16:50:35.401674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401509+0000","last_change":"2024-09-13T16:50:30.590118+0000","last_active":"2024-09-13T16:50:35.401509+0000","last_peered":"2024-09-13T16:50:35.401509+0000","last_clean":"2024-09-13T16:50:35.401509+0000","last_became_active":"2024-09-13T16:50:30.589880+0000","last_became_peered":"2024-09-13T16:50:30.589880+0000","last_unstale":"2024-09-13T16:50:35.401509+0000","last_undegraded":"2024-09-13T16:50:35.401509+0000","last_fullsized":"2024-09-13T16:50:35.401509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:32.908616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087743+0000","last_change":"2024-09-13T16:50:30.103427+0000","last_active":"2024-09-13T16:50:35.087743+0000","last_peered":"2024-09-13T16:50:35.087743+0000","last_clean":"2024-09-13T16:50:35.087743+0000","last_became_active":"2024-09-13T16:50:30.102964+0000","last_became_peered":"2024-09-13T16:50:30.102964+0000","last_unstale":"2024-09-13T16:50:35.087743+0000","last_undegraded":"2024-09-13T16:50:35.087743+0000","last_fullsized":"2024-09-13T16:50:35.087743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402109+0000","last_change":"2024-09-13T16:50:34.632823+0000","last_active":"2024-09-13T16:50:35.402109+0000","last_peered":"2024-09-13T16:50:35.402109+0000","last_clean":"2024-09-13T16:50:35.402109+0000","last_became_active":"2024-09-13T16:50:34.632261+0000","last_became_peered":"2024-09-13T16:50:34.632261+0000","last_unstale":"2024-09-13T16:50:35.402109+0000","last_undegraded":"2024-09-13T16:50:35.402109+0000","last_fullsized":"2024-09-13T16:50:35.402109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462419+0000","last_change":"2024-09-13T16:50:30.606638+0000","last_active":"2024-09-13T16:50:35.462419+0000","last_peered":"2024-09-13T16:50:35.462419+0000","last_clean":"2024-09-13T16:50:35.462419+0000","last_became_active":"2024-09-13T16:50:30.606375+0000","last_became_peered":"2024-09-13T16:50:30.606375+0000","last_unstale":"2024-09-13T16:50:35.462419+0000","last_undegraded":"2024-09-13T16:50:35.462419+0000","last_fullsized":"2024-09-13T16:50:35.462419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:27.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498015+0000","last_change":"2024-09-13T16:50:30.115629+0000","last_active":"2024-09-13T16:50:35.498015+0000","last_peered":"2024-09-13T16:50:35.498015+0000","last_clean":"2024-09-13T16:50:35.498015+0000","last_became_active":"2024-09-13T16:50:30.115336+0000","last_became_peered":"2024-09-13T16:50:30.115336+0000","last_unstale":"2024-09-13T16:50:35.498015+0000","last_undegraded":"2024-09-13T16:50:35.498015+0000","last_fullsized":"2024-09-13T16:50:35.498015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:47.419945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509728+0000","last_change":"2024-09-13T16:50:35.507866+0000","last_active":"2024-09-13T16:50:35.509728+0000","last_peered":"2024-09-13T16:50:35.509728+0000","last_clean":"2024-09-13T16:50:35.509728+0000","last_became_active":"2024-09-13T16:50:35.505548+0000","last_became_peered":"2024-09-13T16:50:35.505548+0000","last_unstale":"2024-09-13T16:50:35.509728+0000","last_undegraded":"2024-09-13T16:50:35.509728+0000","last_fullsized":"2024-09-13T16:50:35.509728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.455646+0000","last_change":"2024-09-13T16:50:30.596915+0000","last_active":"2024-09-13T16:50:35.455646+0000","last_peered":"2024-09-13T16:50:35.455646+0000","last_clean":"2024-09-13T16:50:35.455646+0000","last_became_active":"2024-09-13T16:50:30.596643+0000","last_became_peered":"2024-09-13T16:50:30.596643+0000","last_unstale":"2024-09-13T16:50:35.455646+0000","last_undegraded":"2024-09-13T16:50:35.455646+0000","last_fullsized":"2024-09-13T16:50:35.455646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092229+0000","last_change":"2024-09-13T16:50:30.115172+0000","last_active":"2024-09-13T16:50:35.092229+0000","last_peered":"2024-09-13T16:50:35.092229+0000","last_clean":"2024-09-13T16:50:35.092229+0000","last_became_active":"2024-09-13T16:50:30.114633+0000","last_became_peered":"2024-09-13T16:50:30.114633+0000","last_unstale":"2024-09-13T16:50:35.092229+0000","last_undegraded":"2024-09-13T16:50:35.092229+0000","last_fullsized":"2024-09-13T16:50:35.092229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498080+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:50:35.498080+0000","last_peered":"2024-09-13T16:50:35.498080+0000","last_clean":"2024-09-13T16:50:35.498080+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:50:35.498080+0000","last_undegraded":"2024-09-13T16:50:35.498080+0000","last_fullsized":"2024-09-13T16:50:35.498080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.506121+0000","last_change":"2024-09-13T16:50:35.502219+0000","last_active":"2024-09-13T16:50:35.506121+0000","last_peered":"2024-09-13T16:50:35.506121+0000","last_clean":"2024-09-13T16:50:35.506121+0000","last_became_active":"2024-09-13T16:50:35.501877+0000","last_became_peered":"2024-09-13T16:50:35.501877+0000","last_unstale":"2024-09-13T16:50:35.506121+0000","last_undegraded":"2024-09-13T16:50:35.506121+0000","last_fullsized":"2024-09-13T16:50:35.506121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:37.440533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500614+0000","last_change":"2024-09-13T16:50:30.604143+0000","last_active":"2024-09-13T16:50:35.500614+0000","last_peered":"2024-09-13T16:50:35.500614+0000","last_clean":"2024-09-13T16:50:35.500614+0000","last_became_active":"2024-09-13T16:50:30.603882+0000","last_became_peered":"2024-09-13T16:50:30.603882+0000","last_unstale":"2024-09-13T16:50:35.500614+0000","last_undegraded":"2024-09-13T16:50:35.500614+0000","last_fullsized":"2024-09-13T16:50:35.500614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:31.996725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087857+0000","last_change":"2024-09-13T16:50:30.116691+0000","last_active":"2024-09-13T16:50:35.087857+0000","last_peered":"2024-09-13T16:50:35.087857+0000","last_clean":"2024-09-13T16:50:35.087857+0000","last_became_active":"2024-09-13T16:50:30.115262+0000","last_became_peered":"2024-09-13T16:50:30.115262+0000","last_unstale":"2024-09-13T16:50:35.087857+0000","last_undegraded":"2024-09-13T16:50:35.087857+0000","last_fullsized":"2024-09-13T16:50:35.087857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.506161+0000","last_change":"2024-09-13T16:50:35.499612+0000","last_active":"2024-09-13T16:50:35.506161+0000","last_peered":"2024-09-13T16:50:35.506161+0000","last_clean":"2024-09-13T16:50:35.506161+0000","last_became_active":"2024-09-13T16:50:35.498882+0000","last_became_peered":"2024-09-13T16:50:35.498882+0000","last_unstale":"2024-09-13T16:50:35.506161+0000","last_undegraded":"2024-09-13T16:50:35.506161+0000","last_fullsized":"2024-09-13T16:50:35.506161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:14.360209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092696+0000","last_change":"2024-09-13T16:50:30.601337+0000","last_active":"2024-09-13T16:50:35.092696+0000","last_peered":"2024-09-13T16:50:35.092696+0000","last_clean":"2024-09-13T16:50:35.092696+0000","last_became_active":"2024-09-13T16:50:30.601073+0000","last_became_peered":"2024-09-13T16:50:30.601073+0000","last_unstale":"2024-09-13T16:50:35.092696+0000","last_undegraded":"2024-09-13T16:50:35.092696+0000","last_fullsized":"2024-09-13T16:50:35.092696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.117003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402285+0000","last_change":"2024-09-13T16:50:30.109801+0000","last_active":"2024-09-13T16:50:35.402285+0000","last_peered":"2024-09-13T16:50:35.402285+0000","last_clean":"2024-09-13T16:50:35.402285+0000","last_became_active":"2024-09-13T16:50:30.107936+0000","last_became_peered":"2024-09-13T16:50:30.107936+0000","last_unstale":"2024-09-13T16:50:35.402285+0000","last_undegraded":"2024-09-13T16:50:35.402285+0000","last_fullsized":"2024-09-13T16:50:35.402285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462892+0000","last_change":"2024-09-13T16:50:34.640936+0000","last_active":"2024-09-13T16:50:35.462892+0000","last_peered":"2024-09-13T16:50:35.462892+0000","last_clean":"2024-09-13T16:50:35.462892+0000","last_became_active":"2024-09-13T16:50:34.640606+0000","last_became_peered":"2024-09-13T16:50:34.640606+0000","last_unstale":"2024-09-13T16:50:35.462892+0000","last_undegraded":"2024-09-13T16:50:35.462892+0000","last_fullsized":"2024-09-13T16:50:35.462892+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500787+0000","last_change":"2024-09-13T16:50:30.620140+0000","last_active":"2024-09-13T16:50:35.500787+0000","last_peered":"2024-09-13T16:50:35.500787+0000","last_clean":"2024-09-13T16:50:35.500787+0000","last_became_active":"2024-09-13T16:50:30.619960+0000","last_became_peered":"2024-09-13T16:50:30.619960+0000","last_unstale":"2024-09-13T16:50:35.500787+0000","last_undegraded":"2024-09-13T16:50:35.500787+0000","last_fullsized":"2024-09-13T16:50:35.500787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:28.715573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.088162+0000","last_change":"2024-09-13T16:50:30.104959+0000","last_active":"2024-09-13T16:50:35.088162+0000","last_peered":"2024-09-13T16:50:35.088162+0000","last_clean":"2024-09-13T16:50:35.088162+0000","last_became_active":"2024-09-13T16:50:30.104709+0000","last_became_peered":"2024-09-13T16:50:30.104709+0000","last_unstale":"2024-09-13T16:50:35.088162+0000","last_undegraded":"2024-09-13T16:50:35.088162+0000","last_fullsized":"2024-09-13T16:50:35.088162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.828286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.505477+0000","last_change":"2024-09-13T16:50:35.501120+0000","last_active":"2024-09-13T16:50:35.505477+0000","last_peered":"2024-09-13T16:50:35.505477+0000","last_clean":"2024-09-13T16:50:35.505477+0000","last_became_active":"2024-09-13T16:50:35.500500+0000","last_became_peered":"2024-09-13T16:50:35.500500+0000","last_unstale":"2024-09-13T16:50:35.505477+0000","last_undegraded":"2024-09-13T16:50:35.505477+0000","last_fullsized":"2024-09-13T16:50:35.505477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499025+0000","last_change":"2024-09-13T16:50:30.615966+0000","last_active":"2024-09-13T16:50:35.499025+0000","last_peered":"2024-09-13T16:50:35.499025+0000","last_clean":"2024-09-13T16:50:35.499025+0000","last_became_active":"2024-09-13T16:50:30.615750+0000","last_became_peered":"2024-09-13T16:50:30.615750+0000","last_unstale":"2024-09-13T16:50:35.499025+0000","last_undegraded":"2024-09-13T16:50:35.499025+0000","last_fullsized":"2024-09-13T16:50:35.499025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501690+0000","last_change":"2024-09-13T16:50:30.112955+0000","last_active":"2024-09-13T16:50:35.501690+0000","last_peered":"2024-09-13T16:50:35.501690+0000","last_clean":"2024-09-13T16:50:35.501690+0000","last_became_active":"2024-09-13T16:50:30.111633+0000","last_became_peered":"2024-09-13T16:50:30.111633+0000","last_unstale":"2024-09-13T16:50:35.501690+0000","last_undegraded":"2024-09-13T16:50:35.501690+0000","last_fullsized":"2024-09-13T16:50:35.501690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499166+0000","last_change":"2024-09-13T16:50:34.636875+0000","last_active":"2024-09-13T16:50:35.499166+0000","last_peered":"2024-09-13T16:50:35.499166+0000","last_clean":"2024-09-13T16:50:35.499166+0000","last_became_active":"2024-09-13T16:50:34.636350+0000","last_became_peered":"2024-09-13T16:50:34.636350+0000","last_unstale":"2024-09-13T16:50:35.499166+0000","last_undegraded":"2024-09-13T16:50:35.499166+0000","last_fullsized":"2024-09-13T16:50:35.499166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:32.980957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498929+0000","last_change":"2024-09-13T16:50:30.617707+0000","last_active":"2024-09-13T16:50:35.498929+0000","last_peered":"2024-09-13T16:50:35.498929+0000","last_clean":"2024-09-13T16:50:35.498929+0000","last_became_active":"2024-09-13T16:50:30.617279+0000","last_became_peered":"2024-09-13T16:50:30.617279+0000","last_unstale":"2024-09-13T16:50:35.498929+0000","last_undegraded":"2024-09-13T16:50:35.498929+0000","last_fullsized":"2024-09-13T16:50:35.498929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087386+0000","last_change":"2024-09-13T16:50:30.107834+0000","last_active":"2024-09-13T16:50:35.087386+0000","last_peered":"2024-09-13T16:50:35.087386+0000","last_clean":"2024-09-13T16:50:35.087386+0000","last_became_active":"2024-09-13T16:50:30.107457+0000","last_became_peered":"2024-09-13T16:50:30.107457+0000","last_unstale":"2024-09-13T16:50:35.087386+0000","last_undegraded":"2024-09-13T16:50:35.087386+0000","last_fullsized":"2024-09-13T16:50:35.087386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463011+0000","last_change":"2024-09-13T16:50:34.635554+0000","last_active":"2024-09-13T16:50:35.463011+0000","last_peered":"2024-09-13T16:50:35.463011+0000","last_clean":"2024-09-13T16:50:35.463011+0000","last_became_active":"2024-09-13T16:50:34.635194+0000","last_became_peered":"2024-09-13T16:50:34.635194+0000","last_unstale":"2024-09-13T16:50:35.463011+0000","last_undegraded":"2024-09-13T16:50:35.463011+0000","last_fullsized":"2024-09-13T16:50:35.463011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:54.702371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091123+0000","last_change":"2024-09-13T16:50:30.612793+0000","last_active":"2024-09-13T16:50:35.091123+0000","last_peered":"2024-09-13T16:50:35.091123+0000","last_clean":"2024-09-13T16:50:35.091123+0000","last_became_active":"2024-09-13T16:50:30.612243+0000","last_became_peered":"2024-09-13T16:50:30.612243+0000","last_unstale":"2024-09-13T16:50:35.091123+0000","last_undegraded":"2024-09-13T16:50:35.091123+0000","last_fullsized":"2024-09-13T16:50:35.091123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091094+0000","last_change":"2024-09-13T16:50:30.097963+0000","last_active":"2024-09-13T16:50:35.091094+0000","last_peered":"2024-09-13T16:50:35.091094+0000","last_clean":"2024-09-13T16:50:35.091094+0000","last_became_active":"2024-09-13T16:50:30.097015+0000","last_became_peered":"2024-09-13T16:50:30.097015+0000","last_unstale":"2024-09-13T16:50:35.091094+0000","last_undegraded":"2024-09-13T16:50:35.091094+0000","last_fullsized":"2024-09-13T16:50:35.091094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.514416+0000","last_change":"2024-09-13T16:50:35.512427+0000","last_active":"2024-09-13T16:50:35.514416+0000","last_peered":"2024-09-13T16:50:35.514416+0000","last_clean":"2024-09-13T16:50:35.514416+0000","last_became_active":"2024-09-13T16:50:35.512074+0000","last_became_peered":"2024-09-13T16:50:35.512074+0000","last_unstale":"2024-09-13T16:50:35.514416+0000","last_undegraded":"2024-09-13T16:50:35.514416+0000","last_fullsized":"2024-09-13T16:50:35.514416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:37.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498993+0000","last_change":"2024-09-13T16:50:30.617270+0000","last_active":"2024-09-13T16:50:35.498993+0000","last_peered":"2024-09-13T16:50:35.498993+0000","last_clean":"2024-09-13T16:50:35.498993+0000","last_became_active":"2024-09-13T16:50:30.617047+0000","last_became_peered":"2024-09-13T16:50:30.617047+0000","last_unstale":"2024-09-13T16:50:35.498993+0000","last_undegraded":"2024-09-13T16:50:35.498993+0000","last_fullsized":"2024-09-13T16:50:35.498993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091892+0000","last_change":"2024-09-13T16:50:30.109703+0000","last_active":"2024-09-13T16:50:35.091892+0000","last_peered":"2024-09-13T16:50:35.091892+0000","last_clean":"2024-09-13T16:50:35.091892+0000","last_became_active":"2024-09-13T16:50:30.109427+0000","last_became_peered":"2024-09-13T16:50:30.109427+0000","last_unstale":"2024-09-13T16:50:35.091892+0000","last_undegraded":"2024-09-13T16:50:35.091892+0000","last_fullsized":"2024-09-13T16:50:35.091892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509958+0000","last_change":"2024-09-13T16:50:35.501616+0000","last_active":"2024-09-13T16:50:35.509958+0000","last_peered":"2024-09-13T16:50:35.509958+0000","last_clean":"2024-09-13T16:50:35.509958+0000","last_became_active":"2024-09-13T16:50:35.500854+0000","last_became_peered":"2024-09-13T16:50:35.500854+0000","last_unstale":"2024-09-13T16:50:35.509958+0000","last_undegraded":"2024-09-13T16:50:35.509958+0000","last_fullsized":"2024-09-13T16:50:35.509958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:18.549426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462681+0000","last_change":"2024-09-13T16:50:30.619507+0000","last_active":"2024-09-13T16:50:35.462681+0000","last_peered":"2024-09-13T16:50:35.462681+0000","last_clean":"2024-09-13T16:50:35.462681+0000","last_became_active":"2024-09-13T16:50:30.619122+0000","last_became_peered":"2024-09-13T16:50:30.619122+0000","last_unstale":"2024-09-13T16:50:35.462681+0000","last_undegraded":"2024-09-13T16:50:35.462681+0000","last_fullsized":"2024-09-13T16:50:35.462681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:00.203031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463139+0000","last_change":"2024-09-13T16:50:30.099206+0000","last_active":"2024-09-13T16:50:35.463139+0000","last_peered":"2024-09-13T16:50:35.463139+0000","last_clean":"2024-09-13T16:50:35.463139+0000","last_became_active":"2024-09-13T16:50:30.098688+0000","last_became_peered":"2024-09-13T16:50:30.098688+0000","last_unstale":"2024-09-13T16:50:35.463139+0000","last_undegraded":"2024-09-13T16:50:35.463139+0000","last_fullsized":"2024-09-13T16:50:35.463139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.505292+0000","last_change":"2024-09-13T16:50:35.502406+0000","last_active":"2024-09-13T16:50:35.505292+0000","last_peered":"2024-09-13T16:50:35.505292+0000","last_clean":"2024-09-13T16:50:35.505292+0000","last_became_active":"2024-09-13T16:50:35.502014+0000","last_became_peered":"2024-09-13T16:50:35.502014+0000","last_unstale":"2024-09-13T16:50:35.505292+0000","last_undegraded":"2024-09-13T16:50:35.505292+0000","last_fullsized":"2024-09-13T16:50:35.505292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498570+0000","last_change":"2024-09-13T16:50:30.613053+0000","last_active":"2024-09-13T16:50:35.498570+0000","last_peered":"2024-09-13T16:50:35.498570+0000","last_clean":"2024-09-13T16:50:35.498570+0000","last_became_active":"2024-09-13T16:50:30.612775+0000","last_became_peered":"2024-09-13T16:50:30.612775+0000","last_unstale":"2024-09-13T16:50:35.498570+0000","last_undegraded":"2024-09-13T16:50:35.498570+0000","last_fullsized":"2024-09-13T16:50:35.498570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:54.244738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T16:50:47.108380+0000","last_change":"2024-09-13T16:50:30.111095+0000","last_active":"2024-09-13T16:50:47.108380+0000","last_peered":"2024-09-13T16:50:47.108380+0000","last_clean":"2024-09-13T16:50:47.108380+0000","last_became_active":"2024-09-13T16:50:30.110886+0000","last_became_peered":"2024-09-13T16:50:30.110886+0000","last_unstale":"2024-09-13T16:50:47.108380+0000","last_undegraded":"2024-09-13T16:50:47.108380+0000","last_fullsized":"2024-09-13T16:50:47.108380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401424+0000","last_change":"2024-09-13T16:50:34.641283+0000","last_active":"2024-09-13T16:50:35.401424+0000","last_peered":"2024-09-13T16:50:35.401424+0000","last_clean":"2024-09-13T16:50:35.401424+0000","last_became_active":"2024-09-13T16:50:34.636369+0000","last_became_peered":"2024-09-13T16:50:34.636369+0000","last_unstale":"2024-09-13T16:50:35.401424+0000","last_undegraded":"2024-09-13T16:50:35.401424+0000","last_fullsized":"2024-09-13T16:50:35.401424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:46.586048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.502226+0000","last_change":"2024-09-13T16:50:30.611541+0000","last_active":"2024-09-13T16:50:35.502226+0000","last_peered":"2024-09-13T16:50:35.502226+0000","last_clean":"2024-09-13T16:50:35.502226+0000","last_became_active":"2024-09-13T16:50:30.611046+0000","last_became_peered":"2024-09-13T16:50:30.611046+0000","last_unstale":"2024-09-13T16:50:35.502226+0000","last_undegraded":"2024-09-13T16:50:35.502226+0000","last_fullsized":"2024-09-13T16:50:35.502226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:51.423821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402184+0000","last_change":"2024-09-13T16:50:30.111117+0000","last_active":"2024-09-13T16:50:35.402184+0000","last_peered":"2024-09-13T16:50:35.402184+0000","last_clean":"2024-09-13T16:50:35.402184+0000","last_became_active":"2024-09-13T16:50:30.110857+0000","last_became_peered":"2024-09-13T16:50:30.110857+0000","last_unstale":"2024-09-13T16:50:35.402184+0000","last_undegraded":"2024-09-13T16:50:35.402184+0000","last_fullsized":"2024-09-13T16:50:35.402184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500960+0000","last_change":"2024-09-13T16:50:34.634925+0000","last_active":"2024-09-13T16:50:35.500960+0000","last_peered":"2024-09-13T16:50:35.500960+0000","last_clean":"2024-09-13T16:50:35.500960+0000","last_became_active":"2024-09-13T16:50:34.634329+0000","last_became_peered":"2024-09-13T16:50:34.634329+0000","last_unstale":"2024-09-13T16:50:35.500960+0000","last_undegraded":"2024-09-13T16:50:35.500960+0000","last_fullsized":"2024-09-13T16:50:35.500960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:30.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463375+0000","last_change":"2024-09-13T16:50:30.609227+0000","last_active":"2024-09-13T16:50:35.463375+0000","last_peered":"2024-09-13T16:50:35.463375+0000","last_clean":"2024-09-13T16:50:35.463375+0000","last_became_active":"2024-09-13T16:50:30.608863+0000","last_became_peered":"2024-09-13T16:50:30.608863+0000","last_unstale":"2024-09-13T16:50:35.463375+0000","last_undegraded":"2024-09-13T16:50:35.463375+0000","last_fullsized":"2024-09-13T16:50:35.463375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093039+0000","last_change":"2024-09-13T16:50:30.109198+0000","last_active":"2024-09-13T16:50:35.093039+0000","last_peered":"2024-09-13T16:50:35.093039+0000","last_clean":"2024-09-13T16:50:35.093039+0000","last_became_active":"2024-09-13T16:50:30.108896+0000","last_became_peered":"2024-09-13T16:50:30.108896+0000","last_unstale":"2024-09-13T16:50:35.093039+0000","last_undegraded":"2024-09-13T16:50:35.093039+0000","last_fullsized":"2024-09-13T16:50:35.093039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456883+0000","last_change":"2024-09-13T16:50:34.630350+0000","last_active":"2024-09-13T16:50:35.456883+0000","last_peered":"2024-09-13T16:50:35.456883+0000","last_clean":"2024-09-13T16:50:35.456883+0000","last_became_active":"2024-09-13T16:50:34.626215+0000","last_became_peered":"2024-09-13T16:50:34.626215+0000","last_unstale":"2024-09-13T16:50:35.456883+0000","last_undegraded":"2024-09-13T16:50:35.456883+0000","last_fullsized":"2024-09-13T16:50:35.456883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:51.705201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092550+0000","last_change":"2024-09-13T16:50:30.603528+0000","last_active":"2024-09-13T16:50:35.092550+0000","last_peered":"2024-09-13T16:50:35.092550+0000","last_clean":"2024-09-13T16:50:35.092550+0000","last_became_active":"2024-09-13T16:50:30.603265+0000","last_became_peered":"2024-09-13T16:50:30.603265+0000","last_unstale":"2024-09-13T16:50:35.092550+0000","last_undegraded":"2024-09-13T16:50:35.092550+0000","last_fullsized":"2024-09-13T16:50:35.092550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093205+0000","last_change":"2024-09-13T16:50:30.112126+0000","last_active":"2024-09-13T16:50:35.093205+0000","last_peered":"2024-09-13T16:50:35.093205+0000","last_clean":"2024-09-13T16:50:35.093205+0000","last_became_active":"2024-09-13T16:50:30.111844+0000","last_became_peered":"2024-09-13T16:50:30.111844+0000","last_unstale":"2024-09-13T16:50:35.093205+0000","last_undegraded":"2024-09-13T16:50:35.093205+0000","last_fullsized":"2024-09-13T16:50:35.093205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.511284+0000","last_change":"2024-09-13T16:50:35.507739+0000","last_active":"2024-09-13T16:50:35.511284+0000","last_peered":"2024-09-13T16:50:35.511284+0000","last_clean":"2024-09-13T16:50:35.511284+0000","last_became_active":"2024-09-13T16:50:35.505030+0000","last_became_peered":"2024-09-13T16:50:35.505030+0000","last_unstale":"2024-09-13T16:50:35.511284+0000","last_undegraded":"2024-09-13T16:50:35.511284+0000","last_fullsized":"2024-09-13T16:50:35.511284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:29.316344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509712+0000","last_change":"2024-09-13T16:50:35.506793+0000","last_active":"2024-09-13T16:50:35.509712+0000","last_peered":"2024-09-13T16:50:35.509712+0000","last_clean":"2024-09-13T16:50:35.509712+0000","last_became_active":"2024-09-13T16:50:35.505380+0000","last_became_peered":"2024-09-13T16:50:35.505380+0000","last_unstale":"2024-09-13T16:50:35.509712+0000","last_undegraded":"2024-09-13T16:50:35.509712+0000","last_fullsized":"2024-09-13T16:50:35.509712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:58.754567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.400877+0000","last_change":"2024-09-13T16:50:30.601660+0000","last_active":"2024-09-13T16:50:35.400877+0000","last_peered":"2024-09-13T16:50:35.400877+0000","last_clean":"2024-09-13T16:50:35.400877+0000","last_became_active":"2024-09-13T16:50:30.601315+0000","last_became_peered":"2024-09-13T16:50:30.601315+0000","last_unstale":"2024-09-13T16:50:35.400877+0000","last_undegraded":"2024-09-13T16:50:35.400877+0000","last_fullsized":"2024-09-13T16:50:35.400877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091503+0000","last_change":"2024-09-13T16:50:30.098656+0000","last_active":"2024-09-13T16:50:35.091503+0000","last_peered":"2024-09-13T16:50:35.091503+0000","last_clean":"2024-09-13T16:50:35.091503+0000","last_became_active":"2024-09-13T16:50:30.098137+0000","last_became_peered":"2024-09-13T16:50:30.098137+0000","last_unstale":"2024-09-13T16:50:35.091503+0000","last_undegraded":"2024-09-13T16:50:35.091503+0000","last_fullsized":"2024-09-13T16:50:35.091503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093095+0000","last_change":"2024-09-13T16:50:34.632360+0000","last_active":"2024-09-13T16:50:35.093095+0000","last_peered":"2024-09-13T16:50:35.093095+0000","last_clean":"2024-09-13T16:50:35.093095+0000","last_became_active":"2024-09-13T16:50:34.631958+0000","last_became_peered":"2024-09-13T16:50:34.631958+0000","last_unstale":"2024-09-13T16:50:35.093095+0000","last_undegraded":"2024-09-13T16:50:35.093095+0000","last_fullsized":"2024-09-13T16:50:35.093095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:30.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456997+0000","last_change":"2024-09-13T16:50:30.609936+0000","last_active":"2024-09-13T16:50:35.456997+0000","last_peered":"2024-09-13T16:50:35.456997+0000","last_clean":"2024-09-13T16:50:35.456997+0000","last_became_active":"2024-09-13T16:50:30.609683+0000","last_became_peered":"2024-09-13T16:50:30.609683+0000","last_unstale":"2024-09-13T16:50:35.456997+0000","last_undegraded":"2024-09-13T16:50:35.456997+0000","last_fullsized":"2024-09-13T16:50:35.456997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:32.908616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456931+0000","last_change":"2024-09-13T16:50:30.103999+0000","last_active":"2024-09-13T16:50:35.456931+0000","last_peered":"2024-09-13T16:50:35.456931+0000","last_clean":"2024-09-13T16:50:35.456931+0000","last_became_active":"2024-09-13T16:50:30.103531+0000","last_became_peered":"2024-09-13T16:50:30.103531+0000","last_unstale":"2024-09-13T16:50:35.456931+0000","last_undegraded":"2024-09-13T16:50:35.456931+0000","last_fullsized":"2024-09-13T16:50:35.456931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.828286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091972+0000","last_change":"2024-09-13T16:50:30.103767+0000","last_active":"2024-09-13T16:50:35.091972+0000","last_peered":"2024-09-13T16:50:35.091972+0000","last_clean":"2024-09-13T16:50:35.091972+0000","last_became_active":"2024-09-13T16:50:30.103519+0000","last_became_peered":"2024-09-13T16:50:30.103519+0000","last_unstale":"2024-09-13T16:50:35.091972+0000","last_undegraded":"2024-09-13T16:50:35.091972+0000","last_fullsized":"2024-09-13T16:50:35.091972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463060+0000","last_change":"2024-09-13T16:50:30.618557+0000","last_active":"2024-09-13T16:50:35.463060+0000","last_peered":"2024-09-13T16:50:35.463060+0000","last_clean":"2024-09-13T16:50:35.463060+0000","last_became_active":"2024-09-13T16:50:30.618308+0000","last_became_peered":"2024-09-13T16:50:30.618308+0000","last_unstale":"2024-09-13T16:50:35.463060+0000","last_undegraded":"2024-09-13T16:50:35.463060+0000","last_fullsized":"2024-09-13T16:50:35.463060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:45.160388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087606+0000","last_change":"2024-09-13T16:50:34.639885+0000","last_active":"2024-09-13T16:50:35.087606+0000","last_peered":"2024-09-13T16:50:35.087606+0000","last_clean":"2024-09-13T16:50:35.087606+0000","last_became_active":"2024-09-13T16:50:34.639399+0000","last_became_peered":"2024-09-13T16:50:34.639399+0000","last_unstale":"2024-09-13T16:50:35.087606+0000","last_undegraded":"2024-09-13T16:50:35.087606+0000","last_fullsized":"2024-09-13T16:50:35.087606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:36.425638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456657+0000","last_change":"2024-09-13T16:50:30.102739+0000","last_active":"2024-09-13T16:50:35.456657+0000","last_peered":"2024-09-13T16:50:35.456657+0000","last_clean":"2024-09-13T16:50:35.456657+0000","last_became_active":"2024-09-13T16:50:30.102448+0000","last_became_peered":"2024-09-13T16:50:30.102448+0000","last_unstale":"2024-09-13T16:50:35.456657+0000","last_undegraded":"2024-09-13T16:50:35.456657+0000","last_fullsized":"2024-09-13T16:50:35.456657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:38.882189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501017+0000","last_change":"2024-09-13T16:50:30.597447+0000","last_active":"2024-09-13T16:50:35.501017+0000","last_peered":"2024-09-13T16:50:35.501017+0000","last_clean":"2024-09-13T16:50:35.501017+0000","last_became_active":"2024-09-13T16:50:30.592891+0000","last_became_peered":"2024-09-13T16:50:30.592891+0000","last_unstale":"2024-09-13T16:50:35.501017+0000","last_undegraded":"2024-09-13T16:50:35.501017+0000","last_fullsized":"2024-09-13T16:50:35.501017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:13.892340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456015+0000","last_change":"2024-09-13T16:50:34.630131+0000","last_active":"2024-09-13T16:50:35.456015+0000","last_peered":"2024-09-13T16:50:35.456015+0000","last_clean":"2024-09-13T16:50:35.456015+0000","last_became_active":"2024-09-13T16:50:34.629601+0000","last_became_peered":"2024-09-13T16:50:34.629601+0000","last_unstale":"2024-09-13T16:50:35.456015+0000","last_undegraded":"2024-09-13T16:50:35.456015+0000","last_fullsized":"2024-09-13T16:50:35.456015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:38.058431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.090962+0000","last_change":"2024-09-13T16:50:30.612982+0000","last_active":"2024-09-13T16:50:35.090962+0000","last_peered":"2024-09-13T16:50:35.090962+0000","last_clean":"2024-09-13T16:50:35.090962+0000","last_became_active":"2024-09-13T16:50:30.612577+0000","last_became_peered":"2024-09-13T16:50:30.612577+0000","last_unstale":"2024-09-13T16:50:35.090962+0000","last_undegraded":"2024-09-13T16:50:35.090962+0000","last_fullsized":"2024-09-13T16:50:35.090962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087762+0000","last_change":"2024-09-13T16:50:30.594530+0000","last_active":"2024-09-13T16:50:35.087762+0000","last_peered":"2024-09-13T16:50:35.087762+0000","last_clean":"2024-09-13T16:50:35.087762+0000","last_became_active":"2024-09-13T16:50:30.594248+0000","last_became_peered":"2024-09-13T16:50:30.594248+0000","last_unstale":"2024-09-13T16:50:35.087762+0000","last_undegraded":"2024-09-13T16:50:35.087762+0000","last_fullsized":"2024-09-13T16:50:35.087762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:22.950313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091814+0000","last_change":"2024-09-13T16:50:30.602412+0000","last_active":"2024-09-13T16:50:35.091814+0000","last_peered":"2024-09-13T16:50:35.091814+0000","last_clean":"2024-09-13T16:50:35.091814+0000","last_became_active":"2024-09-13T16:50:30.602146+0000","last_became_peered":"2024-09-13T16:50:30.602146+0000","last_unstale":"2024-09-13T16:50:35.091814+0000","last_undegraded":"2024-09-13T16:50:35.091814+0000","last_fullsized":"2024-09-13T16:50:35.091814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:35.586398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462908+0000","last_change":"2024-09-13T16:50:30.606003+0000","last_active":"2024-09-13T16:50:35.462908+0000","last_peered":"2024-09-13T16:50:35.462908+0000","last_clean":"2024-09-13T16:50:35.462908+0000","last_became_active":"2024-09-13T16:50:30.605562+0000","last_became_peered":"2024-09-13T16:50:30.605562+0000","last_unstale":"2024-09-13T16:50:35.462908+0000","last_undegraded":"2024-09-13T16:50:35.462908+0000","last_fullsized":"2024-09-13T16:50:35.462908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456431+0000","last_change":"2024-09-13T16:50:30.617701+0000","last_active":"2024-09-13T16:50:35.456431+0000","last_peered":"2024-09-13T16:50:35.456431+0000","last_clean":"2024-09-13T16:50:35.456431+0000","last_became_active":"2024-09-13T16:50:30.616581+0000","last_became_peered":"2024-09-13T16:50:30.616581+0000","last_unstale":"2024-09-13T16:50:35.456431+0000","last_undegraded":"2024-09-13T16:50:35.456431+0000","last_fullsized":"2024-09-13T16:50:35.456431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:52.041529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401854+0000","last_change":"2024-09-13T16:50:30.599260+0000","last_active":"2024-09-13T16:50:35.401854+0000","last_peered":"2024-09-13T16:50:35.401854+0000","last_clean":"2024-09-13T16:50:35.401854+0000","last_became_active":"2024-09-13T16:50:30.599018+0000","last_became_peered":"2024-09-13T16:50:30.599018+0000","last_unstale":"2024-09-13T16:50:35.401854+0000","last_undegraded":"2024-09-13T16:50:35.401854+0000","last_fullsized":"2024-09-13T16:50:35.401854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093088+0000","last_change":"2024-09-13T16:50:30.613429+0000","last_active":"2024-09-13T16:50:35.093088+0000","last_peered":"2024-09-13T16:50:35.093088+0000","last_clean":"2024-09-13T16:50:35.093088+0000","last_became_active":"2024-09-13T16:50:30.613163+0000","last_became_peered":"2024-09-13T16:50:30.613163+0000","last_unstale":"2024-09-13T16:50:35.093088+0000","last_undegraded":"2024-09-13T16:50:35.093088+0000","last_fullsized":"2024-09-13T16:50:35.093088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456736+0000","last_change":"2024-09-13T16:50:30.607903+0000","last_active":"2024-09-13T16:50:35.456736+0000","last_peered":"2024-09-13T16:50:35.456736+0000","last_clean":"2024-09-13T16:50:35.456736+0000","last_became_active":"2024-09-13T16:50:30.607618+0000","last_became_peered":"2024-09-13T16:50:30.607618+0000","last_unstale":"2024-09-13T16:50:35.456736+0000","last_undegraded":"2024-09-13T16:50:35.456736+0000","last_fullsized":"2024-09-13T16:50:35.456736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091481+0000","last_change":"2024-09-13T16:50:30.603058+0000","last_active":"2024-09-13T16:50:35.091481+0000","last_peered":"2024-09-13T16:50:35.091481+0000","last_clean":"2024-09-13T16:50:35.091481+0000","last_became_active":"2024-09-13T16:50:30.602556+0000","last_became_peered":"2024-09-13T16:50:30.602556+0000","last_unstale":"2024-09-13T16:50:35.091481+0000","last_undegraded":"2024-09-13T16:50:35.091481+0000","last_fullsized":"2024-09-13T16:50:35.091481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456543+0000","last_change":"2024-09-13T16:50:30.599518+0000","last_active":"2024-09-13T16:50:35.456543+0000","last_peered":"2024-09-13T16:50:35.456543+0000","last_clean":"2024-09-13T16:50:35.456543+0000","last_became_active":"2024-09-13T16:50:30.599242+0000","last_became_peered":"2024-09-13T16:50:30.599242+0000","last_unstale":"2024-09-13T16:50:35.456543+0000","last_undegraded":"2024-09-13T16:50:35.456543+0000","last_fullsized":"2024-09-13T16:50:35.456543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091768+0000","last_change":"2024-09-13T16:50:30.610239+0000","last_active":"2024-09-13T16:50:35.091768+0000","last_peered":"2024-09-13T16:50:35.091768+0000","last_clean":"2024-09-13T16:50:35.091768+0000","last_became_active":"2024-09-13T16:50:30.609978+0000","last_became_peered":"2024-09-13T16:50:30.609978+0000","last_unstale":"2024-09-13T16:50:35.091768+0000","last_undegraded":"2024-09-13T16:50:35.091768+0000","last_fullsized":"2024-09-13T16:50:35.091768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:28.715573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462797+0000","last_change":"2024-09-13T16:50:30.618976+0000","last_active":"2024-09-13T16:50:35.462797+0000","last_peered":"2024-09-13T16:50:35.462797+0000","last_clean":"2024-09-13T16:50:35.462797+0000","last_became_active":"2024-09-13T16:50:30.618644+0000","last_became_peered":"2024-09-13T16:50:30.618644+0000","last_unstale":"2024-09-13T16:50:35.462797+0000","last_undegraded":"2024-09-13T16:50:35.462797+0000","last_fullsized":"2024-09-13T16:50:35.462797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:52.041529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092551+0000","last_change":"2024-09-13T16:50:30.609748+0000","last_active":"2024-09-13T16:50:35.092551+0000","last_peered":"2024-09-13T16:50:35.092551+0000","last_clean":"2024-09-13T16:50:35.092551+0000","last_became_active":"2024-09-13T16:50:30.609504+0000","last_became_peered":"2024-09-13T16:50:30.609504+0000","last_unstale":"2024-09-13T16:50:35.092551+0000","last_undegraded":"2024-09-13T16:50:35.092551+0000","last_fullsized":"2024-09-13T16:50:35.092551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500181+0000","last_change":"2024-09-13T16:50:30.618417+0000","last_active":"2024-09-13T16:50:35.500181+0000","last_peered":"2024-09-13T16:50:35.500181+0000","last_clean":"2024-09-13T16:50:35.500181+0000","last_became_active":"2024-09-13T16:50:30.618114+0000","last_became_peered":"2024-09-13T16:50:30.618114+0000","last_unstale":"2024-09-13T16:50:35.500181+0000","last_undegraded":"2024-09-13T16:50:35.500181+0000","last_fullsized":"2024-09-13T16:50:35.500181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:25.758134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.088098+0000","last_change":"2024-09-13T16:50:30.604041+0000","last_active":"2024-09-13T16:50:35.088098+0000","last_peered":"2024-09-13T16:50:35.088098+0000","last_clean":"2024-09-13T16:50:35.088098+0000","last_became_active":"2024-09-13T16:50:30.599336+0000","last_became_peered":"2024-09-13T16:50:30.599336+0000","last_unstale":"2024-09-13T16:50:35.088098+0000","last_undegraded":"2024-09-13T16:50:35.088098+0000","last_fullsized":"2024-09-13T16:50:35.088098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:13.892340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093370+0000","last_change":"2024-09-13T16:50:30.610847+0000","last_active":"2024-09-13T16:50:35.093370+0000","last_peered":"2024-09-13T16:50:35.093370+0000","last_clean":"2024-09-13T16:50:35.093370+0000","last_became_active":"2024-09-13T16:50:30.610581+0000","last_became_peered":"2024-09-13T16:50:30.610581+0000","last_unstale":"2024-09-13T16:50:35.093370+0000","last_undegraded":"2024-09-13T16:50:35.093370+0000","last_fullsized":"2024-09-13T16:50:35.093370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:21.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091388+0000","last_change":"2024-09-13T16:50:30.618144+0000","last_active":"2024-09-13T16:50:35.091388+0000","last_peered":"2024-09-13T16:50:35.091388+0000","last_clean":"2024-09-13T16:50:35.091388+0000","last_became_active":"2024-09-13T16:50:30.617861+0000","last_became_peered":"2024-09-13T16:50:30.617861+0000","last_unstale":"2024-09-13T16:50:35.091388+0000","last_undegraded":"2024-09-13T16:50:35.091388+0000","last_fullsized":"2024-09-13T16:50:35.091388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.400884+0000","last_change":"2024-09-13T16:50:30.590573+0000","last_active":"2024-09-13T16:50:35.400884+0000","last_peered":"2024-09-13T16:50:35.400884+0000","last_clean":"2024-09-13T16:50:35.400884+0000","last_became_active":"2024-09-13T16:50:30.590338+0000","last_became_peered":"2024-09-13T16:50:30.590338+0000","last_unstale":"2024-09-13T16:50:35.400884+0000","last_undegraded":"2024-09-13T16:50:35.400884+0000","last_fullsized":"2024-09-13T16:50:35.400884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462638+0000","last_change":"2024-09-13T16:50:30.615260+0000","last_active":"2024-09-13T16:50:35.462638+0000","last_peered":"2024-09-13T16:50:35.462638+0000","last_clean":"2024-09-13T16:50:35.462638+0000","last_became_active":"2024-09-13T16:50:30.614529+0000","last_became_peered":"2024-09-13T16:50:30.614529+0000","last_unstale":"2024-09-13T16:50:35.462638+0000","last_undegraded":"2024-09-13T16:50:35.462638+0000","last_fullsized":"2024-09-13T16:50:35.462638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.455914+0000","last_change":"2024-09-13T16:50:30.603994+0000","last_active":"2024-09-13T16:50:35.455914+0000","last_peered":"2024-09-13T16:50:35.455914+0000","last_clean":"2024-09-13T16:50:35.455914+0000","last_became_active":"2024-09-13T16:50:30.603610+0000","last_became_peered":"2024-09-13T16:50:30.603610+0000","last_unstale":"2024-09-13T16:50:35.455914+0000","last_undegraded":"2024-09-13T16:50:35.455914+0000","last_fullsized":"2024-09-13T16:50:35.455914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.502132+0000","last_change":"2024-09-13T16:50:30.603873+0000","last_active":"2024-09-13T16:50:35.502132+0000","last_peered":"2024-09-13T16:50:35.502132+0000","last_clean":"2024-09-13T16:50:35.502132+0000","last_became_active":"2024-09-13T16:50:30.603571+0000","last_became_peered":"2024-09-13T16:50:30.603571+0000","last_unstale":"2024-09-13T16:50:35.502132+0000","last_undegraded":"2024-09-13T16:50:35.502132+0000","last_fullsized":"2024-09-13T16:50:35.502132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:22.950313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092808+0000","last_change":"2024-09-13T16:50:30.607038+0000","last_active":"2024-09-13T16:50:35.092808+0000","last_peered":"2024-09-13T16:50:35.092808+0000","last_clean":"2024-09-13T16:50:35.092808+0000","last_became_active":"2024-09-13T16:50:30.606788+0000","last_became_peered":"2024-09-13T16:50:30.606788+0000","last_unstale":"2024-09-13T16:50:35.092808+0000","last_undegraded":"2024-09-13T16:50:35.092808+0000","last_fullsized":"2024-09-13T16:50:35.092808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093030+0000","last_change":"2024-09-13T16:50:30.599427+0000","last_active":"2024-09-13T16:50:35.093030+0000","last_peered":"2024-09-13T16:50:35.093030+0000","last_clean":"2024-09-13T16:50:35.093030+0000","last_became_active":"2024-09-13T16:50:30.599149+0000","last_became_peered":"2024-09-13T16:50:30.599149+0000","last_unstale":"2024-09-13T16:50:35.093030+0000","last_undegraded":"2024-09-13T16:50:35.093030+0000","last_fullsized":"2024-09-13T16:50:35.093030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500732+0000","last_change":"2024-09-13T16:50:30.615710+0000","last_active":"2024-09-13T16:50:35.500732+0000","last_peered":"2024-09-13T16:50:35.500732+0000","last_clean":"2024-09-13T16:50:35.500732+0000","last_became_active":"2024-09-13T16:50:30.615444+0000","last_became_peered":"2024-09-13T16:50:30.615444+0000","last_unstale":"2024-09-13T16:50:35.500732+0000","last_undegraded":"2024-09-13T16:50:35.500732+0000","last_fullsized":"2024-09-13T16:50:35.500732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:35.586398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456590+0000","last_change":"2024-09-13T16:50:30.599700+0000","last_active":"2024-09-13T16:50:35.456590+0000","last_peered":"2024-09-13T16:50:35.456590+0000","last_clean":"2024-09-13T16:50:35.456590+0000","last_became_active":"2024-09-13T16:50:30.599132+0000","last_became_peered":"2024-09-13T16:50:30.599132+0000","last_unstale":"2024-09-13T16:50:35.456590+0000","last_undegraded":"2024-09-13T16:50:35.456590+0000","last_fullsized":"2024-09-13T16:50:35.456590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498763+0000","last_change":"2024-09-13T16:50:30.617544+0000","last_active":"2024-09-13T16:50:35.498763+0000","last_peered":"2024-09-13T16:50:35.498763+0000","last_clean":"2024-09-13T16:50:35.498763+0000","last_became_active":"2024-09-13T16:50:30.617008+0000","last_became_peered":"2024-09-13T16:50:30.617008+0000","last_unstale":"2024-09-13T16:50:35.498763+0000","last_undegraded":"2024-09-13T16:50:35.498763+0000","last_fullsized":"2024-09-13T16:50:35.498763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091720+0000","last_change":"2024-09-13T16:50:30.613899+0000","last_active":"2024-09-13T16:50:35.091720+0000","last_peered":"2024-09-13T16:50:35.091720+0000","last_clean":"2024-09-13T16:50:35.091720+0000","last_became_active":"2024-09-13T16:50:30.613520+0000","last_became_peered":"2024-09-13T16:50:30.613520+0000","last_unstale":"2024-09-13T16:50:35.091720+0000","last_undegraded":"2024-09-13T16:50:35.091720+0000","last_fullsized":"2024-09-13T16:50:35.091720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498780+0000","last_change":"2024-09-13T16:50:30.617169+0000","last_active":"2024-09-13T16:50:35.498780+0000","last_peered":"2024-09-13T16:50:35.498780+0000","last_clean":"2024-09-13T16:50:35.498780+0000","last_became_active":"2024-09-13T16:50:30.616608+0000","last_became_peered":"2024-09-13T16:50:30.616608+0000","last_unstale":"2024-09-13T16:50:35.498780+0000","last_undegraded":"2024-09-13T16:50:35.498780+0000","last_fullsized":"2024-09-13T16:50:35.498780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.117003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092229+0000","last_change":"2024-09-13T16:50:30.601494+0000","last_active":"2024-09-13T16:50:35.092229+0000","last_peered":"2024-09-13T16:50:35.092229+0000","last_clean":"2024-09-13T16:50:35.092229+0000","last_became_active":"2024-09-13T16:50:30.601235+0000","last_became_peered":"2024-09-13T16:50:30.601235+0000","last_unstale":"2024-09-13T16:50:35.092229+0000","last_undegraded":"2024-09-13T16:50:35.092229+0000","last_fullsized":"2024-09-13T16:50:35.092229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:54.244738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463054+0000","last_change":"2024-09-13T16:50:30.611789+0000","last_active":"2024-09-13T16:50:35.463054+0000","last_peered":"2024-09-13T16:50:35.463054+0000","last_clean":"2024-09-13T16:50:35.463054+0000","last_became_active":"2024-09-13T16:50:30.608646+0000","last_became_peered":"2024-09-13T16:50:30.608646+0000","last_unstale":"2024-09-13T16:50:35.463054+0000","last_undegraded":"2024-09-13T16:50:35.463054+0000","last_fullsized":"2024-09-13T16:50:35.463054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.145790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402335+0000","last_change":"2024-09-13T16:50:30.609892+0000","last_active":"2024-09-13T16:50:35.402335+0000","last_peered":"2024-09-13T16:50:35.402335+0000","last_clean":"2024-09-13T16:50:35.402335+0000","last_became_active":"2024-09-13T16:50:30.609664+0000","last_became_peered":"2024-09-13T16:50:30.609664+0000","last_unstale":"2024-09-13T16:50:35.402335+0000","last_undegraded":"2024-09-13T16:50:35.402335+0000","last_fullsized":"2024-09-13T16:50:35.402335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500628+0000","last_change":"2024-09-13T16:50:30.607977+0000","last_active":"2024-09-13T16:50:35.500628+0000","last_peered":"2024-09-13T16:50:35.500628+0000","last_clean":"2024-09-13T16:50:35.500628+0000","last_became_active":"2024-09-13T16:50:30.607735+0000","last_became_peered":"2024-09-13T16:50:30.607735+0000","last_unstale":"2024-09-13T16:50:35.500628+0000","last_undegraded":"2024-09-13T16:50:35.500628+0000","last_fullsized":"2024-09-13T16:50:35.500628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:21.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":864256,"data_stored":697795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":278528,"data_stored":106971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.839}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.913}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8979999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":106845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9139999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":697851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:50:57.560 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T16:50:57.560 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T16:50:57.561 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T16:50:57.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T16:50:57.561 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T16:50:57.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.765+0000 7f896b200700 1 Processor -- start 2024-09-13T16:50:57.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.765+0000 7f896b200700 1 -- start start 2024-09-13T16:50:57.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f896b200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f896c146ee0 0x7f896c1472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f896b200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f896c06c5e0 0x7f896c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f896b200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06fea0 0x7f896c070310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f896b200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896c070a80 con 0x7f896c146ee0 2024-09-13T16:50:57.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f896b200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896c164cc0 con 0x7f896c06c5e0 2024-09-13T16:50:57.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f896b200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c06cf90 con 0x7f896c06fea0 2024-09-13T16:50:57.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f896c146ee0 0x7f896c1472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f896c06c5e0 0x7f896c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f896c146ee0 0x7f896c1472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54408/0 (socket says 172.21.15.3:54408) 2024-09-13T16:50:57.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969e00700 1 -- 172.21.15.3:0/1919644919 learned_addr learned my addr 172.21.15.3:0/1919644919 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f896c06c5e0 0x7f896c06ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:51742/0 (socket says 172.21.15.3:51742) 2024-09-13T16:50:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f896a800700 1 --2- 172.21.15.3:0/1919644919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06fea0 0x7f896c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969400700 1 -- 172.21.15.3:0/1919644919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06fea0 msgr2=0x7f896c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969400700 1 --2- 172.21.15.3:0/1919644919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06fea0 0x7f896c070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969400700 1 -- 172.21.15.3:0/1919644919 >> v2:172.21.15.3:3300/0 conn(0x7f896c146ee0 msgr2=0x7f896c1472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969400700 1 --2- 172.21.15.3:0/1919644919 >> v2:172.21.15.3:3300/0 conn(0x7f896c146ee0 0x7f896c1472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969400700 1 -- 172.21.15.3:0/1919644919 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c070850 con 0x7f896c06c5e0 2024-09-13T16:50:57.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8969400700 1 --2- 172.21.15.3:0/1919644919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c06c5e0 0x7f896c06ca50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f896c06b830 tx=0x7f8958009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f3c3bc8da0b01d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8968a00700 1 -- 172.21.15.3:0/1919644919 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958011830 con 0x7f896c06c5e0 2024-09-13T16:50:57.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8968a00700 1 -- 172.21.15.3:0/1919644919 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8958011990 con 0x7f896c06c5e0 2024-09-13T16:50:57.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.769+0000 7f8968a00700 1 -- 172.21.15.3:0/1919644919 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958011d20 con 0x7f896c06c5e0 2024-09-13T16:50:57.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 -- 172.21.15.3:0/1919644919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c06c5e0 msgr2=0x7f896c06ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 --2- 172.21.15.3:0/1919644919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c06c5e0 0x7f896c06ca50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f896c06b830 tx=0x7f8958009b30 comp rx=0 tx=0).stop 2024-09-13T16:50:57.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 -- 172.21.15.3:0/1919644919 shutdown_connections 2024-09-13T16:50:57.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 --2- 172.21.15.3:0/1919644919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06fea0 0x7f896c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 --2- 172.21.15.3:0/1919644919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c06c5e0 0x7f896c06ca50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 --2- 172.21.15.3:0/1919644919 >> v2:172.21.15.3:3300/0 conn(0x7f896c146ee0 0x7f896c1472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 -- 172.21.15.3:0/1919644919 >> 172.21.15.3:0/1919644919 conn(0x7f896c0b0cf0 msgr2=0x7f896c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:57.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 -- 172.21.15.3:0/1919644919 shutdown_connections 2024-09-13T16:50:57.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.773+0000 7f896b200700 1 -- 172.21.15.3:0/1919644919 wait complete. 2024-09-13T16:50:57.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896b200700 1 Processor -- start 2024-09-13T16:50:57.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896b200700 1 -- start start 2024-09-13T16:50:57.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896b200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06c5e0 0x7f896c13f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896b200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896c06fea0 0x7f896c157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c146ee0 0x7f896c13d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896b200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c13efb0 con 0x7f896c06fea0 2024-09-13T16:50:57.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0b2f20 con 0x7f896c146ee0 2024-09-13T16:50:57.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896b200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c13e330 con 0x7f896c06c5e0 2024-09-13T16:50:57.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896c06fea0 0x7f896c157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896c06fea0 0x7f896c157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54410/0 (socket says 172.21.15.3:54410) 2024-09-13T16:50:57.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 -- 172.21.15.3:0/3694972323 learned_addr learned my addr 172.21.15.3:0/3694972323 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:57.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969e00700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06c5e0 0x7f896c13f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f896a800700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c146ee0 0x7f896c13d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 -- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06c5e0 msgr2=0x7f896c13f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06c5e0 0x7f896c13f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 -- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c146ee0 msgr2=0x7f896c13d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c146ee0 0x7f896c13d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 -- 172.21.15.3:0/3694972323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580094b0 con 0x7f896c06fea0 2024-09-13T16:50:57.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.777+0000 7f8969400700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896c06fea0 0x7f896c157f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8958009b70 tx=0x7f8958004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.781+0000 7f8962000700 1 -- 172.21.15.3:0/3694972323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958004240 con 0x7f896c06fea0 2024-09-13T16:50:57.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.781+0000 7f8962000700 1 -- 172.21.15.3:0/3694972323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895801f070 con 0x7f896c06fea0 2024-09-13T16:50:57.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.781+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c13e610 con 0x7f896c06fea0 2024-09-13T16:50:57.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.781+0000 7f8962000700 1 -- 172.21.15.3:0/3694972323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580045e0 con 0x7f896c06fea0 2024-09-13T16:50:57.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.781+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c13ebd0 con 0x7f896c06fea0 2024-09-13T16:50:57.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.781+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c13f0f0 con 0x7f896c06fea0 2024-09-13T16:50:57.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.781+0000 7f8962000700 1 -- 172.21.15.3:0/3694972323 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8958021030 con 0x7f896c06fea0 2024-09-13T16:50:57.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.785+0000 7f8962000700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f894c066730 0x7f894c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.785+0000 7f8962000700 1 -- 172.21.15.3:0/3694972323 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8958085480 con 0x7f896c06fea0 2024-09-13T16:50:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.785+0000 7f8969e00700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f894c066730 0x7f894c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.785+0000 7f8969e00700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f894c066730 0x7f894c068bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f895c009b50 tx=0x7f895c0092e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:57.789+0000 7f8962000700 1 -- 172.21.15.3:0/3694972323 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f896c13f0f0 con 0x7f896c06fea0 2024-09-13T16:50:58.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.025+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f896c13f0f0 con 0x7f896c06fea0 2024-09-13T16:50:58.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.033+0000 7f8962000700 1 -- 172.21.15.3:0/3694972323 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164906 (secure 0 0 0) 0x7f896c13f0f0 con 0x7f896c06fea0 2024-09-13T16:50:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f894c066730 msgr2=0x7f894c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f894c066730 0x7f894c068bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f895c009b50 tx=0x7f895c0092e0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896c06fea0 msgr2=0x7f896c157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896c06fea0 0x7f896c157f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8958009b70 tx=0x7f8958004770 comp rx=0 tx=0).stop 2024-09-13T16:50:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 shutdown_connections 2024-09-13T16:50:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f894c066730 0x7f894c068bf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c146ee0 0x7f896c13d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896c06fea0 0x7f896c157f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 --2- 172.21.15.3:0/3694972323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896c06c5e0 0x7f896c13f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 >> 172.21.15.3:0/3694972323 conn(0x7f896c0b0cf0 msgr2=0x7f896c0cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 shutdown_connections 2024-09-13T16:50:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.041+0000 7f896b200700 1 -- 172.21.15.3:0/3694972323 wait complete. 2024-09-13T16:50:58.069 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3433838649},{"type":"v1","addr":"172.21.15.157:6833","nonce":3433838649}]},"active_addr":"172.21.15.157:6833/3433838649","active_change":"2024-09-13T16:50:13.162869+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":1648734278}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":2754365195}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":1157624381}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":480100328}]}]} 2024-09-13T16:50:58.074 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T16:50:58.074 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T16:50:58.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.241+0000 7f1643e00700 1 Processor -- start 2024-09-13T16:50:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.241+0000 7f1643e00700 1 -- start start 2024-09-13T16:50:58.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f164415b140 0x7f164406ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f164415bc30 0x7f164406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164406d0a0 con 0x7f164406c5e0 2024-09-13T16:50:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164406d660 con 0x7f164415bc30 2024-09-13T16:50:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164406d380 con 0x7f164415b140 2024-09-13T16:50:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f163a000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f164406c5e0 0x7f164415b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39408/0 (socket says 172.21.15.3:39408) 2024-09-13T16:50:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f163a000700 1 -- 172.21.15.3:0/233469789 learned_addr learned my addr 172.21.15.3:0/233469789 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643400700 1 --2- 172.21.15.3:0/233469789 >> v2:172.21.15.3:3300/0 conn(0x7f164415bc30 0x7f164406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1642a00700 1 --2- 172.21.15.3:0/233469789 >> v2:172.21.15.157:3300/0 conn(0x7f164415b140 0x7f164406ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643400700 1 -- 172.21.15.3:0/233469789 >> v2:172.21.15.157:3300/0 conn(0x7f164415b140 msgr2=0x7f164406ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643400700 1 --2- 172.21.15.3:0/233469789 >> v2:172.21.15.157:3300/0 conn(0x7f164415b140 0x7f164406ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643400700 1 -- 172.21.15.3:0/233469789 >> v1:172.21.15.3:6789/0 conn(0x7f164406c5e0 legacy=0x7f164415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643400700 1 -- 172.21.15.3:0/233469789 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164406a530 con 0x7f164415bc30 2024-09-13T16:50:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1642000700 1 -- 172.21.15.3:0/233469789 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335943327 0 0) 0x7f164406d0a0 con 0x7f164406c5e0 2024-09-13T16:50:58.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1643400700 1 --2- 172.21.15.3:0/233469789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f164406ca70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f16440b1900 tx=0x7f163c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287ee47ba399eb09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1642000700 1 -- 172.21.15.3:0/233469789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c00e070 con 0x7f164415bc30 2024-09-13T16:50:58.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1642000700 1 -- 172.21.15.3:0/233469789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c00bb20 con 0x7f164415bc30 2024-09-13T16:50:58.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.245+0000 7f1642000700 1 -- 172.21.15.3:0/233469789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c012430 con 0x7f164415bc30 2024-09-13T16:50:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.249+0000 7f1643e00700 1 -- 172.21.15.3:0/233469789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 msgr2=0x7f164406ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.249+0000 7f1643e00700 1 --2- 172.21.15.3:0/233469789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f164406ca70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f16440b1900 tx=0x7f163c00b330 comp rx=0 tx=0).stop 2024-09-13T16:50:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.249+0000 7f1643e00700 1 -- 172.21.15.3:0/233469789 shutdown_connections 2024-09-13T16:50:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.249+0000 7f1643e00700 1 --2- 172.21.15.3:0/233469789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f164406ca70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.249+0000 7f1643e00700 1 --2- 172.21.15.3:0/233469789 >> v2:172.21.15.157:3300/0 conn(0x7f164415b140 0x7f164406ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.249+0000 7f1643e00700 1 -- 172.21.15.3:0/233469789 >> 172.21.15.3:0/233469789 conn(0x7f16440b0cf0 msgr2=0x7f16440b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.249+0000 7f1643e00700 1 -- 172.21.15.3:0/233469789 shutdown_connections 2024-09-13T16:50:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.249+0000 7f1643e00700 1 -- 172.21.15.3:0/233469789 wait complete. 2024-09-13T16:50:58.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643e00700 1 Processor -- start 2024-09-13T16:50:58.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643e00700 1 -- start start 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164406c5e0 0x7f1644145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415b140 0x7f1644142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f16440c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440c4b80 con 0x7f164415bc30 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440c9c80 con 0x7f164415b140 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16440c4080 con 0x7f164406c5e0 2024-09-13T16:50:58.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f16440c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f16440c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54428/0 (socket says 172.21.15.3:54428) 2024-09-13T16:50:58.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643400700 1 -- 172.21.15.3:0/1206192001 learned_addr learned my addr 172.21.15.3:0/1206192001 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1642a00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164406c5e0 0x7f1644145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f163a000700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415b140 0x7f1644142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643400700 1 -- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164406c5e0 msgr2=0x7f1644145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643400700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164406c5e0 0x7f1644145710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643400700 1 -- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415b140 msgr2=0x7f1644142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643400700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415b140 0x7f1644142b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f1643400700 1 -- 172.21.15.3:0/1206192001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c009a70 con 0x7f164415bc30 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.253+0000 7f163a000700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415b140 0x7f1644142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.257+0000 7f1643400700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f16440c3b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b680 tx=0x7f163c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.257+0000 7f163be00700 1 -- 172.21.15.3:0/1206192001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c012430 con 0x7f164415bc30 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.257+0000 7f163be00700 1 -- 172.21.15.3:0/1206192001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c020070 con 0x7f164415bc30 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.257+0000 7f163be00700 1 -- 172.21.15.3:0/1206192001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c004280 con 0x7f164415bc30 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.257+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16440c9950 con 0x7f164415bc30 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.257+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16440c45c0 con 0x7f164415bc30 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.257+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164406ad30 con 0x7f164415bc30 2024-09-13T16:50:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.261+0000 7f163be00700 1 -- 172.21.15.3:0/1206192001 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f163c0043e0 con 0x7f164415bc30 2024-09-13T16:50:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.261+0000 7f163be00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1620066730 0x7f1620068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.261+0000 7f163be00700 1 -- 172.21.15.3:0/1206192001 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f163c022070 con 0x7f164415bc30 2024-09-13T16:50:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.261+0000 7f1642a00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1620066730 0x7f1620068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.261+0000 7f1642a00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1620066730 0x7f1620068bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1634009bf0 tx=0x7f1634009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.265+0000 7f163be00700 1 -- 172.21.15.3:0/1206192001 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f163c053100 con 0x7f164415bc30 2024-09-13T16:50:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.457+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16440c9dc0 con 0x7f164415bc30 2024-09-13T16:50:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.457+0000 7f163be00700 1 -- 172.21.15.3:0/1206192001 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7f163c02c090 con 0x7f164415bc30 2024-09-13T16:50:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:58.463 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1620066730 msgr2=0x7f1620068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1620066730 0x7f1620068bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1634009bf0 tx=0x7f1634009240 comp rx=0 tx=0).stop 2024-09-13T16:50:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 msgr2=0x7f16440c3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f16440c3b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b680 tx=0x7f163c004af0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 shutdown_connections 2024-09-13T16:50:58.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1620066730 0x7f1620068bf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f164415bc30 0x7f16440c3b40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415b140 0x7f1644142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 --2- 172.21.15.3:0/1206192001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164406c5e0 0x7f1644145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 >> 172.21.15.3:0/1206192001 conn(0x7f16440b0cf0 msgr2=0x7f164415c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 shutdown_connections 2024-09-13T16:50:58.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.461+0000 7f1643e00700 1 -- 172.21.15.3:0/1206192001 wait complete. 2024-09-13T16:50:58.467 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:50:58.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:50:58.489 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T16:50:58.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:50:58.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.701+0000 7f88f4a00700 1 Processor -- start 2024-09-13T16:50:58.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.701+0000 7f88f4a00700 1 -- start start 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88f4a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88f015aee0 0x7f88f00cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88f4a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f00b34b0 0x7f88f00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88f4a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f00cfbb0 con 0x7f88f00b5570 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88f4a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f015be80 con 0x7f88f015aee0 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88f4a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f015b350 con 0x7f88f00b34b0 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88eea00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88f015aee0 0x7f88f00cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88ef400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f88f00b5570 0x7f88f00b3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37492/0 (socket says 172.21.15.3:37492) 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88ef400700 1 -- 172.21.15.3:0/2664560080 learned_addr learned my addr 172.21.15.3:0/2664560080 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88efe00700 1 --2- 172.21.15.3:0/2664560080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f00b34b0 0x7f88f00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88ee000700 1 -- 172.21.15.3:0/2664560080 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774514861 0 0) 0x7f88f00cfbb0 con 0x7f88f00b5570 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88ee000700 1 -- 172.21.15.3:0/2664560080 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d40035d0 con 0x7f88f00b5570 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88efe00700 1 -- 172.21.15.3:0/2664560080 >> v2:172.21.15.3:3300/0 conn(0x7f88f015aee0 msgr2=0x7f88f00cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88efe00700 1 --2- 172.21.15.3:0/2664560080 >> v2:172.21.15.3:3300/0 conn(0x7f88f015aee0 0x7f88f00cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88efe00700 1 -- 172.21.15.3:0/2664560080 >> v1:172.21.15.157:6789/0 conn(0x7f88f00b5570 legacy=0x7f88f00b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.705+0000 7f88efe00700 1 -- 172.21.15.3:0/2664560080 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f015b5e0 con 0x7f88f00b34b0 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88efe00700 1 --2- 172.21.15.3:0/2664560080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f00b34b0 0x7f88f00b3940 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009d90 tx=0x7f88e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dadb17c9609bc3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88ee000700 1 -- 172.21.15.3:0/2664560080 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800e070 con 0x7f88f00b34b0 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88ee000700 1 -- 172.21.15.3:0/2664560080 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e800bb20 con 0x7f88f00b34b0 2024-09-13T16:50:58.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88ee000700 1 -- 172.21.15.3:0/2664560080 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8012430 con 0x7f88f00b34b0 2024-09-13T16:50:58.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 -- 172.21.15.3:0/2664560080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f00b34b0 msgr2=0x7f88f00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 --2- 172.21.15.3:0/2664560080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f00b34b0 0x7f88f00b3940 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009d90 tx=0x7f88e800b330 comp rx=0 tx=0).stop 2024-09-13T16:50:58.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 -- 172.21.15.3:0/2664560080 shutdown_connections 2024-09-13T16:50:58.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 --2- 172.21.15.3:0/2664560080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f00b34b0 0x7f88f00b3940 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 --2- 172.21.15.3:0/2664560080 >> v2:172.21.15.3:3300/0 conn(0x7f88f015aee0 0x7f88f00cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 -- 172.21.15.3:0/2664560080 >> 172.21.15.3:0/2664560080 conn(0x7f88f0071dc0 msgr2=0x7f88f006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 -- 172.21.15.3:0/2664560080 shutdown_connections 2024-09-13T16:50:58.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 -- 172.21.15.3:0/2664560080 wait complete. 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 Processor -- start 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.709+0000 7f88f4a00700 1 -- start start 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88f4a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f00b34b0 0x7f88f014e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88f4a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f00b5570 0x7f88f0157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88f4a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f015aee0 0x7f88f0158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88f4a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0151cf0 con 0x7f88f00b34b0 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88f4a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0155530 con 0x7f88f00b5570 2024-09-13T16:50:58.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88f4a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f00c9d80 con 0x7f88f015aee0 2024-09-13T16:50:58.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f00b34b0 0x7f88f014e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f00b34b0 0x7f88f014e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54446/0 (socket says 172.21.15.3:54446) 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 -- 172.21.15.3:0/478441273 learned_addr learned my addr 172.21.15.3:0/478441273 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88efe00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f015aee0 0x7f88f0158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88eea00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f00b5570 0x7f88f0157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 -- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f015aee0 msgr2=0x7f88f0158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f015aee0 0x7f88f0158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 -- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f00b5570 msgr2=0x7f88f0157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f00b5570 0x7f88f0157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 -- 172.21.15.3:0/478441273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8009a70 con 0x7f88f00b34b0 2024-09-13T16:50:58.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88ef400700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f00b34b0 0x7f88f014e810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c700 tx=0x7f88e000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88e7400700 1 -- 172.21.15.3:0/478441273 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0005d90 con 0x7f88f00b34b0 2024-09-13T16:50:58.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88e7400700 1 -- 172.21.15.3:0/478441273 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e00070d0 con 0x7f88f00b34b0 2024-09-13T16:50:58.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f00ca060 con 0x7f88f00b34b0 2024-09-13T16:50:58.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88e7400700 1 -- 172.21.15.3:0/478441273 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0007480 con 0x7f88f00b34b0 2024-09-13T16:50:58.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.713+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f0155a60 con 0x7f88f00b34b0 2024-09-13T16:50:58.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.717+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f0155c80 con 0x7f88f00b34b0 2024-09-13T16:50:58.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.717+0000 7f88e7400700 1 -- 172.21.15.3:0/478441273 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88e00075e0 con 0x7f88f00b34b0 2024-09-13T16:50:58.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.717+0000 7f88e7400700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f88d0066730 0x7f88d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.717+0000 7f88e7400700 1 -- 172.21.15.3:0/478441273 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f88e0083910 con 0x7f88f00b34b0 2024-09-13T16:50:58.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.721+0000 7f88eea00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f88d0066730 0x7f88d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.721+0000 7f88eea00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f88d0066730 0x7f88d0068bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f88f0154e80 tx=0x7f88dc009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.729+0000 7f88e7400700 1 -- 172.21.15.3:0/478441273 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f88e0050900 con 0x7f88f00b34b0 2024-09-13T16:50:58.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.925+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88f00b4110 con 0x7f88f00b34b0 2024-09-13T16:50:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.929+0000 7f88e7400700 1 -- 172.21.15.3:0/478441273 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7f88e0053f20 con 0x7f88f00b34b0 2024-09-13T16:50:58.934 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:50:58.934 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:50:58.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f88d0066730 msgr2=0x7f88d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f88d0066730 0x7f88d0068bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f88f0154e80 tx=0x7f88dc009240 comp rx=0 tx=0).stop 2024-09-13T16:50:58.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f00b34b0 msgr2=0x7f88f014e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f00b34b0 0x7f88f014e810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c700 tx=0x7f88e000ca10 comp rx=0 tx=0).stop 2024-09-13T16:50:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 shutdown_connections 2024-09-13T16:50:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f88d0066730 0x7f88d0068bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f015aee0 0x7f88f0158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f00b5570 0x7f88f0157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 --2- 172.21.15.3:0/478441273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f00b34b0 0x7f88f014e810 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 >> 172.21.15.3:0/478441273 conn(0x7f88f0071dc0 msgr2=0x7f88f006cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 shutdown_connections 2024-09-13T16:50:58.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:58.933+0000 7f88f4a00700 1 -- 172.21.15.3:0/478441273 wait complete. 2024-09-13T16:50:58.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T16:50:58.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T16:50:58.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T16:50:58.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T16:50:58.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T16:50:58.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T16:50:58.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T16:50:58.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T16:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.193+0000 7f256a200700 1 Processor -- start 2024-09-13T16:50:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.193+0000 7fbb33e00700 1 Processor -- start 2024-09-13T16:50:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.193+0000 7fbb33e00700 1 -- start start 2024-09-13T16:50:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.193+0000 7f256a200700 1 -- start start 2024-09-13T16:50:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f5d03e00700 1 Processor -- start 2024-09-13T16:50:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f256a200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f256406ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f256a200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f256415b5f0 0x7f256415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f9e7d200700 1 Processor -- start 2024-09-13T16:50:59.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f5d03e00700 1 -- start start 2024-09-13T16:50:59.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f256a200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256406d6b0 con 0x7f256415b210 2024-09-13T16:50:59.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f9e7d200700 1 -- start start 2024-09-13T16:50:59.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f256a200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256406d990 con 0x7f256415b5f0 2024-09-13T16:50:59.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f256a200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256406dc70 con 0x7f256406e750 2024-09-13T16:50:59.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f2563400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f256415b210 0x7f25640b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37496/0 (socket says 172.21.15.3:37496) 2024-09-13T16:50:59.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f2563400700 1 -- 172.21.15.3:0/3114035156 learned_addr learned my addr 172.21.15.3:0/3114035156 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb33e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 0x7fbb3406d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb33e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3415aa00 con 0x7fbb340b4b80 2024-09-13T16:50:59.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb33e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3415ace0 con 0x7fbb3406d9b0 2024-09-13T16:50:59.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb33e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340cdf70 con 0x7fbb340b47a0 2024-09-13T16:50:59.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb32000700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbb3406d9b0 0x7fbb340ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37504/0 (socket says 172.21.15.3:37504) 2024-09-13T16:50:59.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb32000700 1 -- 172.21.15.3:0/412791963 learned_addr learned my addr 172.21.15.3:0/412791963 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7f2563e00700 1 --2- 172.21.15.3:0/3114035156 >> v2:172.21.15.157:3300/0 conn(0x7f256415b5f0 0x7f256415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228187038 0 0) 0x7fbb3415ace0 con 0x7fbb3406d9b0 2024-09-13T16:50:59.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb180035d0 con 0x7fbb3406d9b0 2024-09-13T16:50:59.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227441894 0 0) 0x7fbb3415aa00 con 0x7fbb340b4b80 2024-09-13T16:50:59.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb3415ace0 con 0x7fbb340b4b80 2024-09-13T16:50:59.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.197+0000 7fbb32a00700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 0x7fbb3406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3455466487 0 0) 0x7fbb3415ace0 con 0x7fbb340b4b80 2024-09-13T16:50:59.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 msgr2=0x7fbb3406d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 0x7fbb3406d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 >> v1:172.21.15.157:6789/0 conn(0x7fbb3406d9b0 legacy=0x7fbb340ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340cdb90 con 0x7fbb340b4b80 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fbb24001650 con 0x7fbb340b4b80 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 >> v1:172.21.15.3:6789/0 conn(0x7fbb340b4b80 legacy=0x7fbb341593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb18003c10 0x7fbb18003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 0x7fbb18004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb18004db0 0x7fbb1800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20001230 con 0x7fbb18003c10 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340cdf70 con 0x7fbb18004db0 2024-09-13T16:50:59.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2562a00700 1 --2- 172.21.15.3:0/3114035156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f256406ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2563e00700 1 -- 172.21.15.3:0/3114035156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 msgr2=0x7f256406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2563e00700 1 --2- 172.21.15.3:0/3114035156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f256406ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2563e00700 1 -- 172.21.15.3:0/3114035156 >> v1:172.21.15.157:6789/0 conn(0x7f256415b210 legacy=0x7f25640b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2563e00700 1 -- 172.21.15.3:0/3114035156 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256406df50 con 0x7f256415b5f0 2024-09-13T16:50:59.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e7d200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e7815cbc0 0x7f9e780adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e7d200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e7815cfa0 0x7f9e780ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e7d200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7806a390 con 0x7f9e78070080 2024-09-13T16:50:59.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e7d200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7806a670 con 0x7f9e7815cfa0 2024-09-13T16:50:59.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e7d200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7806a950 con 0x7f9e7815cbc0 2024-09-13T16:50:59.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d0406c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d040cea30 0x7f5d0406a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d040cd280 con 0x7f5d0406c740 2024-09-13T16:50:59.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d040cd080 con 0x7f5d040cea30 2024-09-13T16:50:59.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040cd560 con 0x7f5d04160f10 2024-09-13T16:50:59.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180054e0 con 0x7fbb340b47a0 2024-09-13T16:50:59.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d02a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d0406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2563e00700 1 --2- 172.21.15.3:0/3114035156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256415b5f0 0x7f256415ba80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2554009850 tx=0x7f2554009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dba12bbd75fc0f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d02a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d0406c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52512/0 (socket says 172.21.15.3:52512) 2024-09-13T16:50:59.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d02a00700 1 -- 172.21.15.3:0/3485412043 learned_addr learned my addr 172.21.15.3:0/3485412043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb32000700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 0x7fbb18004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb32a00700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb18003c10 0x7fbb18003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e77400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e7815cbc0 0x7f9e780adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e77400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e7815cbc0 0x7f9e780adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:51812/0 (socket says 172.21.15.3:51812) 2024-09-13T16:50:59.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e77400700 1 -- 172.21.15.3:0/3909344825 learned_addr learned my addr 172.21.15.3:0/3909344825 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb32000700 1 -- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb18004db0 msgr2=0x7fbb1800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb32000700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb18004db0 0x7fbb1800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb32000700 1 -- 172.21.15.3:0/412791963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb18003c10 msgr2=0x7fbb18003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb32000700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb18003c10 0x7fbb18003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2562000700 1 -- 172.21.15.3:0/3114035156 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554011830 con 0x7f256415b5f0 2024-09-13T16:50:59.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2562000700 1 -- 172.21.15.3:0/3114035156 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2554011990 con 0x7f256415b5f0 2024-09-13T16:50:59.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03400700 1 --2- 172.21.15.3:0/3485412043 >> v2:172.21.15.157:3300/0 conn(0x7f5d040cea30 0x7f5d0406a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f2562000700 1 -- 172.21.15.3:0/3114035156 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554011d20 con 0x7f256415b5f0 2024-09-13T16:50:59.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb32000700 1 -- 172.21.15.3:0/412791963 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3415ace0 con 0x7fbb340b47a0 2024-09-13T16:50:59.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e76a00700 1 --2- 172.21.15.3:0/3909344825 >> v2:172.21.15.3:3300/0 conn(0x7f9e7815cfa0 0x7f9e780ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7fbb32000700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 0x7fbb18004870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c1b0 tx=0x7fbb180058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b683d304373ec79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e76a00700 1 -- 172.21.15.3:0/3909344825 >> v2:172.21.15.157:3300/0 conn(0x7f9e7815cbc0 msgr2=0x7f9e780adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e76a00700 1 --2- 172.21.15.3:0/3909344825 >> v2:172.21.15.157:3300/0 conn(0x7f9e7815cbc0 0x7f9e780adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e76a00700 1 -- 172.21.15.3:0/3909344825 >> v1:172.21.15.3:6789/0 conn(0x7f9e78070080 legacy=0x7f9e780ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f9e76a00700 1 -- 172.21.15.3:0/3909344825 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7806ac30 con 0x7f9e7815cfa0 2024-09-13T16:50:59.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03400700 1 -- 172.21.15.3:0/3485412043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 msgr2=0x7f5d0406c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03400700 1 --2- 172.21.15.3:0/3485412043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d0406c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03400700 1 -- 172.21.15.3:0/3485412043 >> v1:172.21.15.3:6789/0 conn(0x7f5d0406c740 legacy=0x7f5d040ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.201+0000 7f5d03400700 1 -- 172.21.15.3:0/3485412043 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d040cd840 con 0x7f5d040cea30 2024-09-13T16:50:59.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000cd10 con 0x7fbb340b47a0 2024-09-13T16:50:59.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2000ce70 con 0x7fbb340b47a0 2024-09-13T16:50:59.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb31600700 1 -- 172.21.15.3:0/412791963 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20004b50 con 0x7fbb340b47a0 2024-09-13T16:50:59.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fe4f6a00700 1 Processor -- start 2024-09-13T16:50:59.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 -- 172.21.15.3:0/3114035156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256415b5f0 msgr2=0x7f256415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 --2- 172.21.15.3:0/3114035156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256415b5f0 0x7f256415ba80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2554009850 tx=0x7f2554009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:59.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 -- 172.21.15.3:0/3114035156 shutdown_connections 2024-09-13T16:50:59.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 --2- 172.21.15.3:0/3114035156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256415b5f0 0x7f256415ba80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 --2- 172.21.15.3:0/3114035156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f256406ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 -- 172.21.15.3:0/3114035156 >> 172.21.15.3:0/3114035156 conn(0x7f25640b2e30 msgr2=0x7f25640b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03400700 1 --2- 172.21.15.3:0/3485412043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 0x7f5d0406a390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009850 tx=0x7f5cf0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ea69dc01f184fe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 -- 172.21.15.3:0/3114035156 shutdown_connections 2024-09-13T16:50:59.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 -- 172.21.15.3:0/3114035156 wait complete. 2024-09-13T16:50:59.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f9e76a00700 1 --2- 172.21.15.3:0/3909344825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cfa0 0x7f9e780ae010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e7806e140 tx=0x7f9e64009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c3711feb63d75e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d01600700 1 -- 172.21.15.3:0/3485412043 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0011830 con 0x7f5d040cea30 2024-09-13T16:50:59.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22f200700 1 Processor -- start 2024-09-13T16:50:59.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb33e00700 1 -- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 msgr2=0x7fbb18004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb33e00700 1 --2- 172.21.15.3:0/412791963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb340b47a0 0x7fbb18004870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c1b0 tx=0x7fbb180058a0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb33400700 1 -- 172.21.15.3:0/412791963 reap_dead start 2024-09-13T16:50:59.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d01600700 1 -- 172.21.15.3:0/3485412043 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf0011990 con 0x7f5d040cea30 2024-09-13T16:50:59.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d01600700 1 -- 172.21.15.3:0/3485412043 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0011d40 con 0x7f5d040cea30 2024-09-13T16:50:59.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22f200700 1 -- start start 2024-09-13T16:50:59.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03e00700 1 -- 172.21.15.3:0/3485412043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 msgr2=0x7f5d0406a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03e00700 1 --2- 172.21.15.3:0/3485412043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 0x7f5d0406a390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009850 tx=0x7f5cf0009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:59.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03e00700 1 -- 172.21.15.3:0/3485412043 shutdown_connections 2024-09-13T16:50:59.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03e00700 1 --2- 172.21.15.3:0/3485412043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 0x7f5d0406a390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03e00700 1 --2- 172.21.15.3:0/3485412043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d0406c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03e00700 1 -- 172.21.15.3:0/3485412043 >> 172.21.15.3:0/3485412043 conn(0x7f5d04071da0 msgr2=0x7f5d040b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fe4f6a00700 1 -- start start 2024-09-13T16:50:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03e00700 1 -- 172.21.15.3:0/3485412043 shutdown_connections 2024-09-13T16:50:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f5d03e00700 1 -- 172.21.15.3:0/3485412043 wait complete. 2024-09-13T16:50:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 Processor -- start 2024-09-13T16:50:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb33e00700 1 -- 172.21.15.3:0/412791963 shutdown_connections 2024-09-13T16:50:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb33e00700 1 -- 172.21.15.3:0/412791963 >> 172.21.15.3:0/412791963 conn(0x7fbb34071da0 msgr2=0x7fbb340b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f256a200700 1 -- start start 2024-09-13T16:50:59.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f00e6a00700 1 Processor -- start 2024-09-13T16:50:59.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7f00e6a00700 1 -- start start 2024-09-13T16:50:59.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fbb33e00700 1 -- 172.21.15.3:0/412791963 shutdown_connections 2024-09-13T16:50:59.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f256a200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f2564154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f256a200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f256415b210 0x7f25641eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f256a200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256415b5f0 0x7f25640c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f256a200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2564154e70 con 0x7f256415b210 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f256a200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25641ef270 con 0x7f256415b5f0 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f256a200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25640c6440 con 0x7f256406e750 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256415b5f0 0x7f25640c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f2564154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f2564154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52540/0 (socket says 172.21.15.3:52540) 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 -- 172.21.15.3:0/1153462335 learned_addr learned my addr 172.21.15.3:0/1153462335 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e76000700 1 -- 172.21.15.3:0/3909344825 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64004230 con 0x7f9e7815cfa0 2024-09-13T16:50:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e76000700 1 -- 172.21.15.3:0/3909344825 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64004390 con 0x7f9e7815cfa0 2024-09-13T16:50:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e76000700 1 -- 172.21.15.3:0/3909344825 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e640036c0 con 0x7f9e7815cfa0 2024-09-13T16:50:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f6a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4f006bab0 0x7fe4f0070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f6000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe4f015c820 0x7fe4f00b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39458/0 (socket says 172.21.15.3:39458) 2024-09-13T16:50:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f6000700 1 -- 172.21.15.3:0/1659027068 learned_addr learned my addr 172.21.15.3:0/1659027068 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f6a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f015d070 con 0x7fe4f015c820 2024-09-13T16:50:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1659027068 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f015d350 con 0x7fe4f00cc110 2024-09-13T16:50:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1659027068 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00b1d70 con 0x7fe4f006bab0 2024-09-13T16:50:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f4c00700 1 --2- 172.21.15.3:0/1659027068 >> v2:172.21.15.157:3300/0 conn(0x7fe4f006bab0 0x7fe4f0070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22f200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2300cca20 0x7fb230070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22f200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23006bab0 0x7fb230070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22f200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23006c3a0 con 0x7fb23015a7c0 2024-09-13T16:50:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22f200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb230071110 con 0x7fb2300cca20 2024-09-13T16:50:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22f200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23006c5f0 con 0x7fb23006bab0 2024-09-13T16:50:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00e006d580 0x7f00e00b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e00cca20 0x7f00e00b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e006a930 con 0x7f00e01590d0 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e00b2060 con 0x7f00e006d580 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00b21f0 con 0x7f00e00cca20 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2562a00700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f256415b210 0x7f25641eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 -- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256415b5f0 msgr2=0x7f25640c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256415b5f0 0x7f25640c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 -- 172.21.15.3:0/1153462335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f256415b210 msgr2=0x7f25641eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f256415b210 0x7f25641eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 -- 172.21.15.3:0/1153462335 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25540094b0 con 0x7f256406e750 2024-09-13T16:50:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e7d200700 1 -- 172.21.15.3:0/3909344825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cfa0 msgr2=0x7f9e780ae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e7d200700 1 --2- 172.21.15.3:0/3909344825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cfa0 0x7f9e780ae010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e7806e140 tx=0x7f9e64009b30 comp rx=0 tx=0).stop 2024-09-13T16:50:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f4c00700 1 -- 172.21.15.3:0/1659027068 >> v1:172.21.15.157:6789/0 conn(0x7fe4f00cc110 legacy=0x7fe4f006b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f4c00700 1 -- 172.21.15.3:0/1659027068 >> v1:172.21.15.3:6789/0 conn(0x7fe4f015c820 legacy=0x7fe4f00b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f4c00700 1 -- 172.21.15.3:0/1659027068 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00cb0e0 con 0x7fe4f006bab0 2024-09-13T16:50:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22de00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb23015a7c0 0x7fb23006b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39448/0 (socket says 172.21.15.3:39448) 2024-09-13T16:50:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.205+0000 7fb22de00700 1 -- 172.21.15.3:0/132742543 learned_addr learned my addr 172.21.15.3:0/132742543 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354426878 0 0) 0x7fb23006c3a0 con 0x7fb23015a7c0 2024-09-13T16:50:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2140035d0 con 0x7fb23015a7c0 2024-09-13T16:50:59.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fbb33e00700 1 -- 172.21.15.3:0/412791963 wait complete. 2024-09-13T16:50:59.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22e800700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23006bab0 0x7fb230070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14915261 0 0) 0x7fb2140035d0 con 0x7fb23015a7c0 2024-09-13T16:50:59.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23006bab0 msgr2=0x7fb230070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23006bab0 0x7fb230070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 >> v2:172.21.15.157:3300/0 conn(0x7fb2300cca20 msgr2=0x7fb230070690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 --2- 172.21.15.3:0/132742543 >> v2:172.21.15.157:3300/0 conn(0x7fb2300cca20 0x7fb230070690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23006c8d0 con 0x7fb23015a7c0 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fb218001670 con 0x7fb23015a7c0 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 >> v1:172.21.15.3:6789/0 conn(0x7fb23015a7c0 legacy=0x7fb23006b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214003a20 0x7fb214003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb214004870 0x7fb214004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb214005230 0x7fb21400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230071110 con 0x7fb214004870 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23006c5f0 con 0x7fb214003a20 2024-09-13T16:50:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21400c180 con 0x7fb214005230 2024-09-13T16:50:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fbb33e00700 1 Processor -- start 2024-09-13T16:50:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e00cca20 0x7f00e00b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e00cca20 0x7f00e00b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52564/0 (socket says 172.21.15.3:52564) 2024-09-13T16:50:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6000700 1 -- 172.21.15.3:0/1788221188 learned_addr learned my addr 172.21.15.3:0/1788221188 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e7d200700 1 -- 172.21.15.3:0/3909344825 shutdown_connections 2024-09-13T16:50:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e7d200700 1 --2- 172.21.15.3:0/3909344825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cfa0 0x7f9e780ae010 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e7d200700 1 --2- 172.21.15.3:0/3909344825 >> v2:172.21.15.157:3300/0 conn(0x7f9e7815cbc0 0x7f9e780adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e7d200700 1 -- 172.21.15.3:0/3909344825 >> 172.21.15.3:0/3909344825 conn(0x7f9e780b0cf0 msgr2=0x7f9e780b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e7d200700 1 -- 172.21.15.3:0/3909344825 shutdown_connections 2024-09-13T16:50:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f9e7d200700 1 -- 172.21.15.3:0/3909344825 wait complete. 2024-09-13T16:50:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22de00700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214003a20 0x7fb214003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fb22e800700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb214005230 0x7fb21400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fbb33e00700 1 -- start start 2024-09-13T16:50:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f2563400700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f2564154930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f255000c700 tx=0x7f255000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7fe4f4c00700 1 --2- 172.21.15.3:0/1659027068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f006bab0 0x7fe4f0070050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400aea0 tx=0x7fe4e400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63259dfa68b84635 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e4c00700 1 --2- 172.21.15.3:0/1788221188 >> v2:172.21.15.3:3300/0 conn(0x7f00e006d580 0x7f00e00b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6000700 1 -- 172.21.15.3:0/1788221188 >> v2:172.21.15.3:3300/0 conn(0x7f00e006d580 msgr2=0x7f00e00b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6000700 1 --2- 172.21.15.3:0/1788221188 >> v2:172.21.15.3:3300/0 conn(0x7f00e006d580 0x7f00e00b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6000700 1 -- 172.21.15.3:0/1788221188 >> v1:172.21.15.3:6789/0 conn(0x7f00e01590d0 legacy=0x7f00e006ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.209+0000 7f00e6000700 1 -- 172.21.15.3:0/1788221188 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e006ac10 con 0x7f00e00cca20 2024-09-13T16:50:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550005d90 con 0x7f256406e750 2024-09-13T16:50:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25500070d0 con 0x7f256406e750 2024-09-13T16:50:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550007440 con 0x7f256406e750 2024-09-13T16:50:59.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f0c09600700 1 Processor -- start 2024-09-13T16:50:59.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22e800700 1 -- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214003a20 msgr2=0x7fb214003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22e800700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214003a20 0x7fb214003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22e800700 1 -- 172.21.15.3:0/132742543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb214004870 msgr2=0x7fb214004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03e00700 1 Processor -- start 2024-09-13T16:50:59.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22e800700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb214004870 0x7fb214004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4efe00700 1 -- 172.21.15.3:0/1659027068 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400f070 con 0x7fe4f006bab0 2024-09-13T16:50:59.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4efe00700 1 -- 172.21.15.3:0/1659027068 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e4003a40 con 0x7fe4f006bab0 2024-09-13T16:50:59.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4efe00700 1 -- 172.21.15.3:0/1659027068 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4003dd0 con 0x7fe4f006bab0 2024-09-13T16:50:59.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22e800700 1 -- 172.21.15.3:0/132742543 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2140035d0 con 0x7fb214005230 2024-09-13T16:50:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22de00700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214003a20 0x7fb214003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f00e6000700 1 --2- 172.21.15.3:0/1788221188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e00cca20 0x7f00e00b2b80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00d800d150 tx=0x7f00d800d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b9b65347fbd2e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1659027068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f006bab0 msgr2=0x7fe4f0070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4f6a00700 1 --2- 172.21.15.3:0/1659027068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f006bab0 0x7fe4f0070050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400aea0 tx=0x7fe4e400c330 comp rx=0 tx=0).stop 2024-09-13T16:50:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f0c09600700 1 -- start start 2024-09-13T16:50:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25640c6720 con 0x7f256406e750 2024-09-13T16:50:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25640c6ce0 con 0x7f256406e750 2024-09-13T16:50:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03e00700 1 -- start start 2024-09-13T16:50:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb340b47a0 0x7fbb34141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f256415bfc0 con 0x7f256406e750 2024-09-13T16:50:59.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1659027068 shutdown_connections 2024-09-13T16:50:59.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4f6a00700 1 --2- 172.21.15.3:0/1659027068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f006bab0 0x7fe4f0070050 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1659027068 >> 172.21.15.3:0/1659027068 conn(0x7fe4f00cb620 msgr2=0x7fe4f00cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f00dfe00700 1 -- 172.21.15.3:0/1788221188 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8010070 con 0x7f00e00cca20 2024-09-13T16:50:59.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f9e7d200700 1 Processor -- start 2024-09-13T16:50:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f00dfe00700 1 -- 172.21.15.3:0/1788221188 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d800db70 con 0x7f00e00cca20 2024-09-13T16:50:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f00dfe00700 1 -- 172.21.15.3:0/1788221188 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8003a40 con 0x7f00e00cca20 2024-09-13T16:50:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22e800700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb214005230 0x7fb21400ba30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb228009f00 tx=0x7fb2140058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1a11a6ac91a2deb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228007dc0 con 0x7fb214005230 2024-09-13T16:50:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228014400 con 0x7fb214005230 2024-09-13T16:50:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22d400700 1 -- 172.21.15.3:0/132742543 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228014790 con 0x7fb214005230 2024-09-13T16:50:59.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb3414c6b0 0x7fbb34155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3414e120 0x7fbb3414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f9e7d200700 1 -- start start 2024-09-13T16:50:59.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3414d6d0 con 0x7fbb340b47a0 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3414dc90 con 0x7fbb3414e120 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34141970 con 0x7fbb3414c6b0 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb32000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb3414c6b0 0x7fbb34155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3414e120 0x7fbb3414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb32000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb3414c6b0 0x7fbb34155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52584/0 (socket says 172.21.15.3:52584) 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb32000700 1 -- 172.21.15.3:0/2259921205 learned_addr learned my addr 172.21.15.3:0/2259921205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22f200700 1 -- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb214005230 msgr2=0x7fb21400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22f200700 1 --2- 172.21.15.3:0/132742543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb214005230 0x7fb21400ba30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb228009f00 tx=0x7fb2140058a0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fb22e800700 1 -- 172.21.15.3:0/132742543 reap_dead start 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0406c740 0x7f5d04151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 0x7f5d040b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d040b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040b2cf0 con 0x7f5d0406c740 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04149520 con 0x7f5d040cea30 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04152540 con 0x7f5d04160f10 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33400700 1 -- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb3414c6b0 msgr2=0x7fbb34155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33400700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb3414c6b0 0x7fbb34155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33400700 1 -- 172.21.15.3:0/2259921205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb340b47a0 msgr2=0x7fbb34141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33400700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb340b47a0 0x7fbb34141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fbb33400700 1 -- 172.21.15.3:0/2259921205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34155f70 con 0x7fbb3414e120 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f9e7d200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e78070080 0x7f9e7815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f9e7d200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cbc0 0x7f9e781f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f9e7d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7815cfa0 0x7f9e7815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f9e7d200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e780ab4b0 con 0x7f9e7815cbc0 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f9e7d200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7814afa0 con 0x7f9e7815cfa0 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f9e7d200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7814e1b0 con 0x7f9e78070080 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f0c09600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c041611f0 0x7f0c040b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f0c09600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c040ce0e0 con 0x7f0c040b3e90 2024-09-13T16:50:59.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f0c09600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c040ce3a0 con 0x7f0c0406c4b0 2024-09-13T16:50:59.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f0c09600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c040ce680 con 0x7f0c041611f0 2024-09-13T16:50:59.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e77400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e78070080 0x7f9e7815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e77e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7815cfa0 0x7f9e7815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e76a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cbc0 0x7f9e781f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e77e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7815cfa0 0x7f9e7815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:51884/0 (socket says 172.21.15.3:51884) 2024-09-13T16:50:59.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e77e00700 1 -- 172.21.15.3:0/1106601713 learned_addr learned my addr 172.21.15.3:0/1106601713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1659027068 shutdown_connections 2024-09-13T16:50:59.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1659027068 wait complete. 2024-09-13T16:50:59.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f0c03400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0c0406c4b0 0x7f0c0406c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37518/0 (socket says 172.21.15.3:37518) 2024-09-13T16:50:59.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f0c03400700 1 -- 172.21.15.3:0/1444280275 learned_addr learned my addr 172.21.15.3:0/1444280275 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fbb33400700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3414e120 0x7fbb3414e5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb340cfc00 tx=0x7fbb2400c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d040b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d040b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52588/0 (socket says 172.21.15.3:52588) 2024-09-13T16:50:59.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.213+0000 7f5d03400700 1 -- 172.21.15.3:0/953969686 learned_addr learned my addr 172.21.15.3:0/953969686 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5d02000700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 0x7f5d040b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f00e6a00700 1 -- 172.21.15.3:0/1788221188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e00cca20 msgr2=0x7f00e00b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f00e6a00700 1 --2- 172.21.15.3:0/1788221188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e00cca20 0x7f00e00b2b80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00d800d150 tx=0x7f00d800d460 comp rx=0 tx=0).stop 2024-09-13T16:50:59.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5d02000700 1 -- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 msgr2=0x7f5d040b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5d02000700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d04160f10 0x7f5d040b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5d02000700 1 -- 172.21.15.3:0/953969686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0406c740 msgr2=0x7f5d04151c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:50:59.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fb22f200700 1 -- 172.21.15.3:0/132742543 shutdown_connections 2024-09-13T16:50:59.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fb22f200700 1 -- 172.21.15.3:0/132742543 >> 172.21.15.3:0/132742543 conn(0x7fb2300cb620 msgr2=0x7fb2300cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5d02000700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0406c740 0x7f5d04151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5d02000700 1 -- 172.21.15.3:0/953969686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf00094b0 con 0x7f5d040cea30 2024-09-13T16:50:59.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fb22f200700 1 -- 172.21.15.3:0/132742543 shutdown_connections 2024-09-13T16:50:59.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e76a00700 1 -- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e78070080 msgr2=0x7f9e7815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e76a00700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e78070080 0x7f9e7815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e76a00700 1 -- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7815cfa0 msgr2=0x7f9e7815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e76a00700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7815cfa0 0x7f9e7815a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e76a00700 1 -- 172.21.15.3:0/1106601713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640094b0 con 0x7f9e7815cbc0 2024-09-13T16:50:59.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24004a20 con 0x7fbb3414e120 2024-09-13T16:50:59.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24004b80 con 0x7fbb3414e120 2024-09-13T16:50:59.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb34156250 con 0x7fbb3414e120 2024-09-13T16:50:59.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb34155c80 con 0x7fbb3414e120 2024-09-13T16:50:59.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25500075a0 con 0x7f256406e750 2024-09-13T16:50:59.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fb22f200700 1 -- 172.21.15.3:0/132742543 wait complete. 2024-09-13T16:50:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fe4f6a00700 1 Processor -- start 2024-09-13T16:50:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f0c02a00700 1 --2- 172.21.15.3:0/1444280275 >> v2:172.21.15.157:3300/0 conn(0x7f0c041611f0 0x7f0c040b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fe4f6a00700 1 -- start start 2024-09-13T16:50:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f00e6a00700 1 -- 172.21.15.3:0/1788221188 shutdown_connections 2024-09-13T16:50:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f00e6a00700 1 --2- 172.21.15.3:0/1788221188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e00cca20 0x7f00e00b2b80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f00e6a00700 1 --2- 172.21.15.3:0/1788221188 >> v2:172.21.15.3:3300/0 conn(0x7f00e006d580 0x7f00e00b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f00e6a00700 1 -- 172.21.15.3:0/1788221188 >> 172.21.15.3:0/1788221188 conn(0x7f00e00b0cf0 msgr2=0x7f00e00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f0c01600700 1 -- 172.21.15.3:0/1444280275 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791712251 0 0) 0x7f0c040ce3a0 con 0x7f0c0406c4b0 2024-09-13T16:50:59.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f0c02a00700 1 -- 172.21.15.3:0/1444280275 >> v1:172.21.15.157:6789/0 conn(0x7f0c0406c4b0 legacy=0x7f0c0406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f0c02a00700 1 -- 172.21.15.3:0/1444280275 >> v1:172.21.15.3:6789/0 conn(0x7f0c040b3e90 legacy=0x7f0c0406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f0c02a00700 1 -- 172.21.15.3:0/1444280275 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040ce960 con 0x7f0c041611f0 2024-09-13T16:50:59.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f00e6a00700 1 -- 172.21.15.3:0/1788221188 shutdown_connections 2024-09-13T16:50:59.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f00e6a00700 1 -- 172.21.15.3:0/1788221188 wait complete. 2024-09-13T16:50:59.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5d02000700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 0x7f5d040b36d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d040b2a10 tx=0x7f5cfc00cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e76a00700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cbc0 0x7f9e781f71a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e64015000 tx=0x7f9e640047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5cfaa00700 1 -- 172.21.15.3:0/953969686 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc003c20 con 0x7f5d040cea30 2024-09-13T16:50:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f0c02a00700 1 --2- 172.21.15.3:0/1444280275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 0x7f0c040b3950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009850 tx=0x7f0bec009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8967197879d85ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5cfaa00700 1 -- 172.21.15.3:0/953969686 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc003d80 con 0x7f5d040cea30 2024-09-13T16:50:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f5cfaa00700 1 -- 172.21.15.3:0/953969686 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00ab00 con 0x7f5d040cea30 2024-09-13T16:50:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb34156440 con 0x7fbb3414e120 2024-09-13T16:50:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24007030 con 0x7fbb3414e120 2024-09-13T16:50:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f255b400700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2544066800 0x7f2544068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2550083830 con 0x7f256406e750 2024-09-13T16:50:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f255b400700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7f254406fe40 0x7f2544072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 --> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2544072950 con 0x7f254406fe40 2024-09-13T16:50:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2550085e50 con 0x7f256406e750 2024-09-13T16:50:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64011630 con 0x7f9e7815cbc0 2024-09-13T16:50:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64011790 con 0x7f9e7815cbc0 2024-09-13T16:50:59.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64011b40 con 0x7f9e7815cbc0 2024-09-13T16:50:59.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f006bab0 0x7fe4f00adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4f00cc110 0x7fe4f00ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f015c820 0x7fe4f00ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fe4f6a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0070590 con 0x7fe4f015c820 2024-09-13T16:50:59.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fe4f6a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00acc60 con 0x7fe4f006bab0 2024-09-13T16:50:59.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.217+0000 7fe4f6a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00abbc0 con 0x7fe4f00cc110 2024-09-13T16:50:59.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f2562a00700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2544066800 0x7f2544068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7814d630 con 0x7f9e7815cbc0 2024-09-13T16:50:59.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7814dbf0 con 0x7f9e7815cbc0 2024-09-13T16:50:59.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f2563e00700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7f254406fe40 0x7f2544072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f2562a00700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2544066800 0x7f2544068cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f256415b030 tx=0x7f255c003a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e780ccb60 con 0x7f9e7815cbc0 2024-09-13T16:50:59.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb24007190 con 0x7fbb3414e120 2024-09-13T16:50:59.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fb22f200700 1 Processor -- start 2024-09-13T16:50:59.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4f00cc110 0x7fe4f00ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f00e6a00700 1 Processor -- start 2024-09-13T16:50:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c01600700 1 -- 172.21.15.3:0/1444280275 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec011830 con 0x7f0c041611f0 2024-09-13T16:50:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d04152820 con 0x7f5d040cea30 2024-09-13T16:50:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d040b31b0 con 0x7f5d040cea30 2024-09-13T16:50:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4f00cc110 0x7fe4f00ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52612/0 (socket says 172.21.15.3:52612) 2024-09-13T16:50:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 -- 172.21.15.3:0/1482554159 learned_addr learned my addr 172.21.15.3:0/1482554159 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c01600700 1 -- 172.21.15.3:0/1444280275 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec011990 con 0x7f0c041611f0 2024-09-13T16:50:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c01600700 1 -- 172.21.15.3:0/1444280275 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec011d20 con 0x7f0c041611f0 2024-09-13T16:50:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f6000700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f015c820 0x7fe4f00ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f00e6a00700 1 -- start start 2024-09-13T16:50:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fbb2aa00700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbb14066800 0x7fbb14068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 -- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f006bab0 msgr2=0x7fe4f00adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f006bab0 0x7fe4f00adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 -- 172.21.15.3:0/1482554159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f015c820 msgr2=0x7fe4f00ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f015c820 0x7fe4f00ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 -- 172.21.15.3:0/1482554159 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e400ab80 con 0x7fe4f00cc110 2024-09-13T16:50:59.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fb22f200700 1 -- start start 2024-09-13T16:50:59.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c09600700 1 -- 172.21.15.3:0/1444280275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 msgr2=0x7f0c040b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c09600700 1 --2- 172.21.15.3:0/1444280275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 0x7f0c040b3950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009850 tx=0x7f0bec009b60 comp rx=0 tx=0).stop 2024-09-13T16:50:59.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4f4c00700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4f00cc110 0x7fe4f00ab070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0072350 tx=0x7fe4e4004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fb22f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214005230 0x7fb23015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fb22f200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23014c220 0x7fb23014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbb240844f0 con 0x7fbb3414e120 2024-09-13T16:50:59.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fbb2aa00700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7fbb1406fe40 0x7fbb14072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 --> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb14072950 con 0x7fbb1406fe40 2024-09-13T16:50:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb240848a0 con 0x7fbb3414e120 2024-09-13T16:50:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fbb32a00700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbb14066800 0x7fbb14068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e64011ca0 con 0x7f9e7815cbc0 2024-09-13T16:50:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f9e6f400700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9e58066780 0x7f9e58068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e64086120 con 0x7f9e7815cbc0 2024-09-13T16:50:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4004340 con 0x7fe4f00cc110 2024-09-13T16:50:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e40044a0 con 0x7fe4f00cc110 2024-09-13T16:50:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4004850 con 0x7fe4f00cc110 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f00abea0 con 0x7fe4f00cc110 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f00ac460 con 0x7fe4f00cc110 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4f0070050 con 0x7fe4f00cc110 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb225400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23014c220 0x7fb23014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb225400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23014c220 0x7fb23014ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54530/0 (socket says 172.21.15.3:54530) 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22f200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23014efc0 0x7fb230147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22f200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23014e6a0 con 0x7fb23014c220 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300b2100 con 0x7fb214005230 2024-09-13T16:50:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22f200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300b23a0 con 0x7fb23014efc0 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb225400700 1 -- 172.21.15.3:0/2613267991 learned_addr learned my addr 172.21.15.3:0/2613267991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22e800700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23014efc0 0x7fb230147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22de00700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214005230 0x7fb23015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22e800700 1 -- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214005230 msgr2=0x7fb23015c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22e800700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb214005230 0x7fb23015c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22e800700 1 -- 172.21.15.3:0/2613267991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23014c220 msgr2=0x7fb23014ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22e800700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23014c220 0x7fb23014ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22e800700 1 -- 172.21.15.3:0/2613267991 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300b2680 con 0x7fb23014efc0 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22e800700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23014efc0 0x7fb230147de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb228016000 tx=0x7fb228004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c09600700 1 -- 172.21.15.3:0/1444280275 shutdown_connections 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c09600700 1 --2- 172.21.15.3:0/1444280275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 0x7f0c040b3950 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c09600700 1 -- 172.21.15.3:0/1444280275 >> 172.21.15.3:0/1444280275 conn(0x7f0c040cd7e0 msgr2=0x7f0c04071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f0c09600700 1 -- 172.21.15.3:0/1444280275 shutdown_connections 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f0c09600700 1 -- 172.21.15.3:0/1444280275 wait complete. 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f00e6a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e006d580 0x7f00e01528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f00e6a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00e00cca20 0x7f00e01efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.221+0000 7f00e6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e01590d0 0x7f00e0157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0157e30 con 0x7f00e00cca20 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0158110 con 0x7f00e01590d0 2024-09-13T16:50:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e01583f0 con 0x7f00e006d580 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e01590d0 0x7f00e0157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e01590d0 0x7f00e0157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:51924/0 (socket says 172.21.15.3:51924) 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 -- 172.21.15.3:0/1246926198 learned_addr learned my addr 172.21.15.3:0/1246926198 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 -- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e006d580 msgr2=0x7f00e01528d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00e006d580 0x7f00e01528d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 -- 172.21.15.3:0/1246926198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00e00cca20 msgr2=0x7f00e01efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00e00cca20 0x7f00e01efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 -- 172.21.15.3:0/1246926198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d800bd40 con 0x7f00e01590d0 2024-09-13T16:50:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00e6000700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e01590d0 0x7f00e0157010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00d800d4a0 tx=0x7f00d8004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228024070 con 0x7fb23014efc0 2024-09-13T16:50:59.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228003a40 con 0x7fb23014efc0 2024-09-13T16:50:59.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228007000 con 0x7fb23014efc0 2024-09-13T16:50:59.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300b2960 con 0x7fb23014efc0 2024-09-13T16:50:59.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f0c09600700 1 Processor -- start 2024-09-13T16:50:59.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f9e77400700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9e58066780 0x7f9e58068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f9e6f400700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f9e5806fdc0 0x7f9e58072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 --> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e580728d0 con 0x7f9e5806fdc0 2024-09-13T16:50:59.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e640865c0 con 0x7f9e7815cbc0 2024-09-13T16:50:59.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f5cfaa00700 1 -- 172.21.15.3:0/953969686 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5cfc00ac60 con 0x7f5d040cea30 2024-09-13T16:50:59.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f5cfaa00700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ce4066800 0x7f5ce4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f5cfaa00700 1 -- 172.21.15.3:0/953969686 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5cfc084410 con 0x7f5d040cea30 2024-09-13T16:50:59.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f5d02a00700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ce4066800 0x7f5ce4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f5d03e00700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f5d041534f0 0x7f5d04060db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8004440 con 0x7f00e01590d0 2024-09-13T16:50:59.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 --> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d040b4660 con 0x7f5d041534f0 2024-09-13T16:50:59.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f9e77e00700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f9e5806fdc0 0x7f9e58072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300b3f60 con 0x7fb23014efc0 2024-09-13T16:50:59.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fbb32000700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7fbb1406fe40 0x7fbb14072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fbb32a00700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbb14066800 0x7fbb14068cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009f80 tx=0x7fbb2c009750 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f0c09600700 1 -- start start 2024-09-13T16:50:59.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c09600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0406c4b0 0x7f0c040b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c09600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c040b3e90 0x7f0c040b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c09600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 0x7f0c04152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f9e77400700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9e58066780 0x7f9e58068c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9e7815b610 tx=0x7f9e680092e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f9e77e00700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f9e5806fdc0 0x7f9e58072200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f5d03400700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f5d041534f0 0x7f5d04060db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb23015aba0 con 0x7fb23014efc0 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c02000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c040b3e90 0x7f0c040b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c02000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c040b3e90 0x7f0c040b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54544/0 (socket says 172.21.15.3:54544) 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c02a00700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0406c4b0 0x7f0c040b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c02000700 1 -- 172.21.15.3:0/4193674111 learned_addr learned my addr 172.21.15.3:0/4193674111 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c03400700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 0x7f0c04152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c04151c90 con 0x7f0c040b3e90 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c041520b0 con 0x7f0c041611f0 2024-09-13T16:50:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c04151010 con 0x7f0c0406c4b0 2024-09-13T16:50:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.225+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d80045a0 con 0x7f00e01590d0 2024-09-13T16:50:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8004930 con 0x7f00e01590d0 2024-09-13T16:50:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e01586d0 con 0x7f00e01590d0 2024-09-13T16:50:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e01579a0 con 0x7f00e01590d0 2024-09-13T16:50:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00e0158280 con 0x7f00e01590d0 2024-09-13T16:50:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f5d03400700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f5d041534f0 0x7f5d04060db0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fbb32000700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7fbb1406fe40 0x7fbb14072280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f2563e00700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7f254406fe40 0x7f2544072280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 <== osd.2 v2:172.21.15.3:6808/4113809917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9e580728d0 con 0x7f9e5806fdc0 2024-09-13T16:50:59.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f5d02a00700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ce4066800 0x7f5ce4068cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009b70 tx=0x7f5cf4009630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f5cfaa00700 1 -- 172.21.15.3:0/953969686 <== osd.3 v2:172.21.15.3:6813/1788023973 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5d040b4660 con 0x7f5d041534f0 2024-09-13T16:50:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4e401d450 con 0x7fe4f00cc110 2024-09-13T16:50:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fe4ed600700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4d4066780 0x7fe4d4068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe4e4086750 con 0x7fe4f00cc110 2024-09-13T16:50:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fe4ed600700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7fe4d406fdc0 0x7fe4d4072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 --> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4d40728d0 con 0x7fe4d406fdc0 2024-09-13T16:50:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e4086ac0 con 0x7fe4f00cc110 2024-09-13T16:50:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fe4f5600700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4d4066780 0x7fe4d4068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb22801e610 con 0x7fb23014efc0 2024-09-13T16:50:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c03400700 1 -- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0406c4b0 msgr2=0x7f0c040b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c03400700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0406c4b0 0x7f0c040b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c03400700 1 -- 172.21.15.3:0/4193674111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c040b3e90 msgr2=0x7f0c040b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c03400700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c040b3e90 0x7f0c040b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c03400700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0094b0 con 0x7f0c041611f0 2024-09-13T16:50:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.229+0000 7f0c02000700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c040b3e90 0x7f0c040b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:59.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fe4f5600700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4d4066780 0x7fe4d4068c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8001700 tx=0x7fe4e80079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f0c03400700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 0x7f0c04152430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400caf0 tx=0x7f0bf400ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fe4f6000700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7fe4d406fdc0 0x7fe4d4072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb226a00700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb20c066730 0x7fb20c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb228086170 con 0x7fb23014efc0 2024-09-13T16:50:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb226a00700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7fb20c06fc70 0x7fb20c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 <== osd.1 v2:172.21.15.3:6800/4234138253 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2544072950 con 0x7f254406fe40 2024-09-13T16:50:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 --> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb20c072780 con 0x7fb20c06fc70 2024-09-13T16:50:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2280864e0 con 0x7fb23014efc0 2024-09-13T16:50:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400a9e0 con 0x7f0c041611f0 2024-09-13T16:50:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb22de00700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb20c066730 0x7fb20c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb225400700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7fb20c06fc70 0x7fb20c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf400ab60 con 0x7f0c041611f0 2024-09-13T16:50:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40070d0 con 0x7f0c041611f0 2024-09-13T16:50:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 <== osd.0 v2:172.21.15.3:6824/3820836136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbb14072950 con 0x7fbb1406fe40 2024-09-13T16:50:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fe4f6000700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7fe4d406fdc0 0x7fe4d4072200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb22de00700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb20c066730 0x7fb20c068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb218000fd0 tx=0x7fb218009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fb225400700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7fb20c06fc70 0x7fb20c0720b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c041512f0 con 0x7f0c041611f0 2024-09-13T16:50:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c041518b0 con 0x7f0c041611f0 2024-09-13T16:50:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c0406dbc0 con 0x7f0c041611f0 2024-09-13T16:50:59.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00d801d040 con 0x7f00e01590d0 2024-09-13T16:50:59.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f00dd600700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f00c006ac10 0x7f00c006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f00d8085290 con 0x7f00e01590d0 2024-09-13T16:50:59.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f00dd600700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7f00c00740e0 0x7f00c0076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 --> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00c0076bf0 con 0x7f00c00740e0 2024-09-13T16:50:59.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d8085640 con 0x7f00e01590d0 2024-09-13T16:50:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.233+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 <== osd.4 v2:172.21.15.157:6808/960819469 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe4d40728d0 con 0x7fe4d406fdc0 2024-09-13T16:50:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f00e5600700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f00c006ac10 0x7f00c006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f00e4c00700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7f00c00740e0 0x7f00c0076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f00e5600700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f00c006ac10 0x7f00c006d0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f00d4003dc0 tx=0x7f00d4007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 <== osd.5 v2:172.21.15.157:6800/2560288896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb20c072780 con 0x7fb20c06fc70 2024-09-13T16:50:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f00e4c00700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7f00c00740e0 0x7f00c0076520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0bf4007230 con 0x7f0c041611f0 2024-09-13T16:50:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f0bfaa00700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0be0066730 0x7f0be0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0bf40841f0 con 0x7f0c041611f0 2024-09-13T16:50:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 <== osd.7 v2:172.21.15.157:6812/3325903215 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f00c0076bf0 con 0x7f00c00740e0 2024-09-13T16:50:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f0c02a00700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0be0066730 0x7f0be0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f0bfaa00700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f0be006fcf0 0x7f0be0072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0be0072800 con 0x7f0be006fcf0 2024-09-13T16:50:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.237+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf4084640 con 0x7f0c041611f0 2024-09-13T16:50:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.241+0000 7f0c02000700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f0be006fcf0 0x7f0be0072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.245+0000 7f0c02a00700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0be0066730 0x7f0be0068bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009fd0 tx=0x7f0bec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.245+0000 7f0c02000700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f0be006fcf0 0x7f0be0072130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.245+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 <== osd.6 v2:172.21.15.157:6824/1906982980 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0be0072800 con 0x7f0be006fcf0 2024-09-13T16:50:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.253+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 --> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d040b33d0 con 0x7f5d041534f0 2024-09-13T16:50:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 --> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e7815d410 con 0x7f9e5806fdc0 2024-09-13T16:50:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5cfaa00700 1 -- 172.21.15.3:0/953969686 <== osd.3 v2:172.21.15.3:6813/1788023973 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5d040b33d0 con 0x7f5d041534f0 2024-09-13T16:50:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e6f400700 1 -- 172.21.15.3:0/1106601713 <== osd.2 v2:172.21.15.3:6808/4113809917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e7815d410 con 0x7f9e5806fdc0 2024-09-13T16:50:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f5d041534f0 msgr2=0x7f5d04060db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03e00700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f5d041534f0 0x7f5d04060db0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ce4066800 msgr2=0x7f5ce4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03e00700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ce4066800 0x7f5ce4068cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009b70 tx=0x7f5cf4009630 comp rx=0 tx=0).stop 2024-09-13T16:50:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 msgr2=0x7f5d040b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03e00700 1 --2- 172.21.15.3:0/953969686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d040cea30 0x7f5d040b36d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d040b2a10 tx=0x7f5cfc00cea0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03400700 1 -- 172.21.15.3:0/953969686 reap_dead start 2024-09-13T16:50:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f9e5806fdc0 msgr2=0x7f9e58072200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e7d200700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f9e5806fdc0 0x7f9e58072200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9e58066780 msgr2=0x7f9e58068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e7d200700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9e58066780 0x7f9e58068c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9e7815b610 tx=0x7f9e680092e0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 shutdown_connections 2024-09-13T16:50:59.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 >> 172.21.15.3:0/953969686 conn(0x7f5d04071da0 msgr2=0x7f5d0415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cbc0 msgr2=0x7f9e781f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e7d200700 1 --2- 172.21.15.3:0/1106601713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e7815cbc0 0x7f9e781f71a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e64015000 tx=0x7f9e640047e0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.257+0000 7f9e77e00700 1 -- 172.21.15.3:0/1106601713 reap_dead start 2024-09-13T16:50:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.261+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 shutdown_connections 2024-09-13T16:50:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.261+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 >> 172.21.15.3:0/1106601713 conn(0x7f9e780b0cf0 msgr2=0x7f9e780aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.261+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 shutdown_connections 2024-09-13T16:50:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.261+0000 7f5d03e00700 1 -- 172.21.15.3:0/953969686 wait complete. 2024-09-13T16:50:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.261+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 shutdown_connections 2024-09-13T16:50:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.261+0000 7f9e7d200700 1 -- 172.21.15.3:0/1106601713 wait complete. 2024-09-13T16:50:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.265+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 --> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb3414ddd0 con 0x7fbb1406fe40 2024-09-13T16:50:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.265+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 --> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb23006ce90 con 0x7fb20c06fc70 2024-09-13T16:50:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.265+0000 7fbb2aa00700 1 -- 172.21.15.3:0/2259921205 <== osd.0 v2:172.21.15.3:6824/3820836136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb3414ddd0 con 0x7fbb1406fe40 2024-09-13T16:50:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.265+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 --> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe4f0072d00 con 0x7fe4d406fdc0 2024-09-13T16:50:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.265+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7fbb1406fe40 msgr2=0x7fbb14072280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.265+0000 7fbb33e00700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7fbb1406fe40 0x7fbb14072280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb226a00700 1 -- 172.21.15.3:0/2613267991 <== osd.5 v2:172.21.15.157:6800/2560288896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb220009560 con 0x7fb20c06fc70 2024-09-13T16:50:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbb14066800 msgr2=0x7fbb14068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33e00700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbb14066800 0x7fbb14068cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009f80 tx=0x7fbb2c009750 comp rx=0 tx=0).stop 2024-09-13T16:50:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3414e120 msgr2=0x7fbb3414e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33e00700 1 --2- 172.21.15.3:0/2259921205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3414e120 0x7fbb3414e5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb340cfc00 tx=0x7fbb2400c7b0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33400700 1 -- 172.21.15.3:0/2259921205 reap_dead start 2024-09-13T16:50:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 shutdown_connections 2024-09-13T16:50:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 >> 172.21.15.3:0/2259921205 conn(0x7fbb34071da0 msgr2=0x7fbb3406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4ed600700 1 -- 172.21.15.3:0/1482554159 <== osd.4 v2:172.21.15.157:6808/960819469 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe4f0072d00 con 0x7fe4d406fdc0 2024-09-13T16:50:59.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 shutdown_connections 2024-09-13T16:50:59.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fbb33e00700 1 -- 172.21.15.3:0/2259921205 wait complete. 2024-09-13T16:50:59.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 --> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2564157740 con 0x7f254406fe40 2024-09-13T16:50:59.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7fe4d406fdc0 msgr2=0x7fe4d4072200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6a00700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7fe4d406fdc0 0x7fe4d4072200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4d4066780 msgr2=0x7fe4d4068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6a00700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4d4066780 0x7fe4d4068c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8001700 tx=0x7fe4e80079e0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4f00cc110 msgr2=0x7fe4f00ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6a00700 1 --2- 172.21.15.3:0/1482554159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4f00cc110 0x7fe4f00ab070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0072350 tx=0x7fe4e4004af0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6000700 1 -- 172.21.15.3:0/1482554159 reap_dead start 2024-09-13T16:50:59.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7f255b400700 1 -- 172.21.15.3:0/1153462335 <== osd.1 v2:172.21.15.3:6800/4234138253 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2564157740 con 0x7f254406fe40 2024-09-13T16:50:59.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 shutdown_connections 2024-09-13T16:50:59.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 >> 172.21.15.3:0/1482554159 conn(0x7fe4f00cb620 msgr2=0x7fe4f00b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7fb20c06fc70 msgr2=0x7fb20c0720b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7fb20c06fc70 0x7fb20c0720b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb20c066730 msgr2=0x7fb20c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb20c066730 0x7fb20c068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb218000fd0 tx=0x7fb218009040 comp rx=0 tx=0).stop 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23014efc0 msgr2=0x7fb230147de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 --2- 172.21.15.3:0/2613267991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb23014efc0 0x7fb230147de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb228016000 tx=0x7fb228004af0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22e800700 1 -- 172.21.15.3:0/2613267991 reap_dead start 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 shutdown_connections 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 >> 172.21.15.3:0/2613267991 conn(0x7fb2300cb620 msgr2=0x7fb23006da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.269+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 shutdown_connections 2024-09-13T16:50:59.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 shutdown_connections 2024-09-13T16:50:59.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7fe4f6a00700 1 -- 172.21.15.3:0/1482554159 wait complete. 2024-09-13T16:50:59.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7fb22f200700 1 -- 172.21.15.3:0/2613267991 wait complete. 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7f254406fe40 msgr2=0x7f2544072280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7f254406fe40 0x7f2544072280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2544066800 msgr2=0x7f2544068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2544066800 0x7f2544068cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f256415b030 tx=0x7f255c003a40 comp rx=0 tx=0).stop 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 msgr2=0x7f2564154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 --2- 172.21.15.3:0/1153462335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256406e750 0x7f2564154930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f255000c700 tx=0x7f255000ca10 comp rx=0 tx=0).stop 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f2563e00700 1 -- 172.21.15.3:0/1153462335 reap_dead start 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 shutdown_connections 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 >> 172.21.15.3:0/1153462335 conn(0x7f25640b2e30 msgr2=0x7f25640d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 shutdown_connections 2024-09-13T16:50:59.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f256a200700 1 -- 172.21.15.3:0/1153462335 wait complete. 2024-09-13T16:50:59.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.273+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 --> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c04151f00 con 0x7f0be006fcf0 2024-09-13T16:50:59.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 --> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f00e006e0e0 con 0x7f00c00740e0 2024-09-13T16:50:59.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f0bfaa00700 1 -- 172.21.15.3:0/4193674111 <== osd.6 v2:172.21.15.157:6824/1906982980 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0c04151f00 con 0x7f0be006fcf0 2024-09-13T16:50:59.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f0be006fcf0 msgr2=0x7f0be0072130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f0c09600700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f0be006fcf0 0x7f0be0072130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00dd600700 1 -- 172.21.15.3:0/1246926198 <== osd.7 v2:172.21.15.157:6812/3325903215 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f00e006e0e0 con 0x7f00c00740e0 2024-09-13T16:50:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7f00c00740e0 msgr2=0x7f00c0076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7f00c00740e0 0x7f00c0076520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f00c006ac10 msgr2=0x7f00c006d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f00c006ac10 0x7f00c006d0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f00d4003dc0 tx=0x7f00d4007a60 comp rx=0 tx=0).stop 2024-09-13T16:50:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e01590d0 msgr2=0x7f00e0157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 --2- 172.21.15.3:0/1246926198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e01590d0 0x7f00e0157010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00d800d4a0 tx=0x7f00d8004af0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6000700 1 -- 172.21.15.3:0/1246926198 reap_dead start 2024-09-13T16:50:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 shutdown_connections 2024-09-13T16:50:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 >> 172.21.15.3:0/1246926198 conn(0x7f00e00b0cf0 msgr2=0x7f00e0153040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 shutdown_connections 2024-09-13T16:50:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.277+0000 7f00e6a00700 1 -- 172.21.15.3:0/1246926198 wait complete. 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0be0066730 msgr2=0x7f0be0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c09600700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0be0066730 0x7f0be0068bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009fd0 tx=0x7f0bec008030 comp rx=0 tx=0).stop 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 msgr2=0x7f0c04152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c09600700 1 --2- 172.21.15.3:0/4193674111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c041611f0 0x7f0c04152430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400caf0 tx=0x7f0bf400ce00 comp rx=0 tx=0).stop 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c03400700 1 -- 172.21.15.3:0/4193674111 reap_dead start 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 shutdown_connections 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 >> 172.21.15.3:0/4193674111 conn(0x7f0c040cd7e0 msgr2=0x7f0c0406ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 shutdown_connections 2024-09-13T16:50:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.281+0000 7f0c09600700 1 -- 172.21.15.3:0/4193674111 wait complete. 2024-09-13T16:50:59.305 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2024-09-13T16:50:59.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T16:50:59.306 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2024-09-13T16:50:59.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T16:50:59.307 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2024-09-13T16:50:59.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T16:50:59.308 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2024-09-13T16:50:59.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T16:50:59.309 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:50:59.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T16:50:59.310 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2024-09-13T16:50:59.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T16:50:59.312 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2024-09-13T16:50:59.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T16:50:59.317 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:50:59.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T16:50:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f47bf200700 1 Processor -- start 2024-09-13T16:50:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f47bf200700 1 -- start start 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.545+0000 7f9ecf400700 1 Processor -- start 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.545+0000 7f9ecf400700 1 -- start start 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ecf400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ecf400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec806d870 con 0x7f9ec8072280 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ecf400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec806e710 con 0x7f9ec80cc920 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ecf400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec806e9f0 con 0x7f9ec8157430 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ec7e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9ec80cc920 0x7f9ec8071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37520/0 (socket says 172.21.15.3:37520) 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ecd400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9ec8072280 0x7f9ec80b2e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39494/0 (socket says 172.21.15.3:39494) 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ec7e00700 1 -- 172.21.15.3:0/2269336532 learned_addr learned my addr 172.21.15.3:0/2269336532 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ecca00700 1 --2- 172.21.15.3:0/2269336532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ec7400700 1 -- 172.21.15.3:0/2269336532 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088169386 0 0) 0x7f9ec806d870 con 0x7f9ec8072280 2024-09-13T16:50:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ec7400700 1 -- 172.21.15.3:0/2269336532 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb00035d0 con 0x7f9ec8072280 2024-09-13T16:50:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f47bf200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47c006cdd0 0x7f47c006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f47bf200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c00cd2b0 con 0x7f47c0142030 2024-09-13T16:50:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f47bf200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c00cd590 con 0x7f47c00b1bd0 2024-09-13T16:50:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f47bf200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c006bd60 con 0x7f47c006cdd0 2024-09-13T16:50:59.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc7c200700 1 Processor -- start 2024-09-13T16:50:59.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ec7400700 1 -- 172.21.15.3:0/2269336532 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247861193 0 0) 0x7f9ec806e710 con 0x7f9ec80cc920 2024-09-13T16:50:59.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ec7400700 1 -- 172.21.15.3:0/2269336532 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec806d870 con 0x7f9ec80cc920 2024-09-13T16:50:59.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ecca00700 1 -- 172.21.15.3:0/2269336532 >> v1:172.21.15.157:6789/0 conn(0x7f9ec80cc920 legacy=0x7f9ec8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.549+0000 7f9ecca00700 1 -- 172.21.15.3:0/2269336532 >> v1:172.21.15.3:6789/0 conn(0x7f9ec8072280 legacy=0x7f9ec80b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47be800700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f47c00b1bd0 0x7f47c00b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37528/0 (socket says 172.21.15.3:37528) 2024-09-13T16:50:59.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47be800700 1 -- 172.21.15.3:0/1056363333 learned_addr learned my addr 172.21.15.3:0/1056363333 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bd400700 1 --2- 172.21.15.3:0/1056363333 >> v2:172.21.15.3:3300/0 conn(0x7f47c006cdd0 0x7f47c006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f9ecca00700 1 -- 172.21.15.3:0/2269336532 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec806ecd0 con 0x7f9ec8157430 2024-09-13T16:50:59.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f9ecca00700 1 --2- 172.21.15.3:0/2269336532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec806dd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009880 tx=0x7f9ebc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a89f27102f0a702 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f8f1be00700 1 Processor -- start 2024-09-13T16:50:59.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc7c200700 1 -- start start 2024-09-13T16:50:59.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bca00700 1 -- 172.21.15.3:0/1056363333 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941657688 0 0) 0x7f47c00cd2b0 con 0x7f47c0142030 2024-09-13T16:50:59.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f9ec7400700 1 -- 172.21.15.3:0/2269336532 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00f8a0 con 0x7f9ec8157430 2024-09-13T16:50:59.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f9ec7400700 1 -- 172.21.15.3:0/2269336532 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc00fa00 con 0x7f9ec8157430 2024-09-13T16:50:59.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f9ec7400700 1 -- 172.21.15.3:0/2269336532 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00fd70 con 0x7f9ec8157430 2024-09-13T16:50:59.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bca00700 1 -- 172.21.15.3:0/1056363333 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a40057e0 con 0x7f47c0142030 2024-09-13T16:50:59.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bd400700 1 -- 172.21.15.3:0/1056363333 >> v1:172.21.15.157:6789/0 conn(0x7f47c00b1bd0 legacy=0x7f47c00b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bd400700 1 -- 172.21.15.3:0/1056363333 >> v1:172.21.15.3:6789/0 conn(0x7f47c0142030 legacy=0x7f47c006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bd400700 1 -- 172.21.15.3:0/1056363333 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c00cd870 con 0x7f47c006cdd0 2024-09-13T16:50:59.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bd400700 1 --2- 172.21.15.3:0/1056363333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 0x7f47c006b820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47b000a9f0 tx=0x7f47b000ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3040e4c52800c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f5ba6c00700 1 Processor -- start 2024-09-13T16:50:59.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc7c200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7406dd60 0x7fbc740b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc7c200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc74070ec0 con 0x7fbc74141d90 2024-09-13T16:50:59.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc7c200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7406c270 con 0x7fbc741419b0 2024-09-13T16:50:59.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc7c200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7406c930 con 0x7fbc7406dd60 2024-09-13T16:50:59.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f8f1be00700 1 -- start start 2024-09-13T16:50:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f5ba6c00700 1 -- start start 2024-09-13T16:50:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc78e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbc74141d90 0x7fbc740b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39522/0 (socket says 172.21.15.3:39522) 2024-09-13T16:50:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc79800700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbc741419b0 0x7fbc74070070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37538/0 (socket says 172.21.15.3:37538) 2024-09-13T16:50:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc79800700 1 -- 172.21.15.3:0/1637011804 learned_addr learned my addr 172.21.15.3:0/1637011804 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7fbc7a200700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7406dd60 0x7fbc740b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bca00700 1 -- 172.21.15.3:0/1056363333 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0004230 con 0x7f47c006cdd0 2024-09-13T16:50:59.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba6c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba00cda60 0x7f5ba00d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bca00700 1 -- 172.21.15.3:0/1056363333 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0004390 con 0x7f47c006cdd0 2024-09-13T16:50:59.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f47bca00700 1 -- 172.21.15.3:0/1056363333 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0004230 con 0x7f47c006cdd0 2024-09-13T16:50:59.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f9ecf400700 1 -- 172.21.15.3:0/2269336532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 msgr2=0x7f9ec806dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.553+0000 7f9ecf400700 1 --2- 172.21.15.3:0/2269336532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec806dd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009880 tx=0x7f9ebc009b90 comp rx=0 tx=0).stop 2024-09-13T16:50:59.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249971753 0 0) 0x7fbc7406c270 con 0x7fbc741419b0 2024-09-13T16:50:59.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc5c0035d0 con 0x7fbc741419b0 2024-09-13T16:50:59.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba6c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ba00b8840 0x7f5ba00d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba5800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba00cda60 0x7f5ba00d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba5800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba00cda60 0x7f5ba00d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52652/0 (socket says 172.21.15.3:52652) 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba5800700 1 -- 172.21.15.3:0/3698503544 learned_addr learned my addr 172.21.15.3:0/3698503544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba6c00700 1 -- 172.21.15.3:0/3698503544 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba00d20b0 con 0x7f5ba015d220 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba6c00700 1 -- 172.21.15.3:0/3698503544 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba00d14a0 con 0x7f5ba00b8840 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3533722424 0 0) 0x7fbc5c0035d0 con 0x7fbc741419b0 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7406dd60 msgr2=0x7fbc740b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7406dd60 0x7fbc740b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 >> v1:172.21.15.3:6789/0 conn(0x7fbc74141d90 legacy=0x7fbc740b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7415fad0 con 0x7fbc741419b0 2024-09-13T16:50:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f47bf200700 1 -- 172.21.15.3:0/1056363333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 msgr2=0x7f47c006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f47bf200700 1 --2- 172.21.15.3:0/1056363333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 0x7f47c006b820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47b000a9f0 tx=0x7f47b000ad00 comp rx=0 tx=0).stop 2024-09-13T16:50:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f1be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c0cca20 0x7f8f1c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f1be00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1c06a930 con 0x7f8f1c06d580 2024-09-13T16:50:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f1be00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1c06ac10 con 0x7f8f1c1590d0 2024-09-13T16:50:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f1be00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0b2060 con 0x7f8f1c0cca20 2024-09-13T16:50:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f1aa00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8f1c1590d0 0x7f8f1c06ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37540/0 (socket says 172.21.15.3:37540) 2024-09-13T16:50:59.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba6c00700 1 -- 172.21.15.3:0/3698503544 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00d1780 con 0x7f5ba00cda60 2024-09-13T16:50:59.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f9ecf400700 1 -- 172.21.15.3:0/2269336532 shutdown_connections 2024-09-13T16:50:59.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f9ecf400700 1 --2- 172.21.15.3:0/2269336532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec806dd00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f9ecf400700 1 -- 172.21.15.3:0/2269336532 >> 172.21.15.3:0/2269336532 conn(0x7f9ec8071050 msgr2=0x7f9ec8071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fbc60001670 con 0x7fbc741419b0 2024-09-13T16:50:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 >> v1:172.21.15.157:6789/0 conn(0x7fbc741419b0 legacy=0x7fbc74070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5c003a20 0x7fbc5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0043f0 0x7fbc5c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc5c0050d0 0x7fbc5c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7406c930 con 0x7fbc5c003a20 2024-09-13T16:50:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c00bfa0 con 0x7fbc5c0043f0 2024-09-13T16:50:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c00c160 con 0x7fbc5c0050d0 2024-09-13T16:50:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f1aa00700 1 -- 172.21.15.3:0/885440098 learned_addr learned my addr 172.21.15.3:0/885440098 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f9ecf400700 1 -- 172.21.15.3:0/2269336532 shutdown_connections 2024-09-13T16:50:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f9ecf400700 1 -- 172.21.15.3:0/2269336532 wait complete. 2024-09-13T16:50:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f47bf200700 1 -- 172.21.15.3:0/1056363333 shutdown_connections 2024-09-13T16:50:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f47bf200700 1 --2- 172.21.15.3:0/1056363333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 0x7f47c006b820 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f47bf200700 1 -- 172.21.15.3:0/1056363333 >> 172.21.15.3:0/1056363333 conn(0x7f47c00b0cf0 msgr2=0x7f47c00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fc985c00700 1 Processor -- start 2024-09-13T16:50:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f1b400700 1 --2- 172.21.15.3:0/885440098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c0cca20 0x7f8f1c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba6200700 1 --2- 172.21.15.3:0/3698503544 >> v2:172.21.15.157:3300/0 conn(0x7f5ba00b8840 0x7f5ba00d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba5800700 1 -- 172.21.15.3:0/3698503544 >> v2:172.21.15.157:3300/0 conn(0x7f5ba00b8840 msgr2=0x7f5ba00d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba5800700 1 --2- 172.21.15.3:0/3698503544 >> v2:172.21.15.157:3300/0 conn(0x7f5ba00b8840 0x7f5ba00d2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba5800700 1 -- 172.21.15.3:0/3698503544 >> v1:172.21.15.3:6789/0 conn(0x7f5ba015d220 legacy=0x7f5ba00cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba5800700 1 -- 172.21.15.3:0/3698503544 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba00d1a60 con 0x7f5ba00cda60 2024-09-13T16:50:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc78e00700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0043f0 0x7fbc5c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc7a200700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc5c0050d0 0x7fbc5c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc79800700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5c003a20 0x7fbc5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fc985c00700 1 -- start start 2024-09-13T16:50:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc78e00700 1 -- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc5c0050d0 msgr2=0x7fbc5c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc78e00700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc5c0050d0 0x7fbc5c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc78e00700 1 -- 172.21.15.3:0/1637011804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5c003a20 msgr2=0x7fbc5c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc78e00700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5c003a20 0x7fbc5c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc78e00700 1 -- 172.21.15.3:0/1637011804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7415fad0 con 0x7fbc5c0043f0 2024-09-13T16:50:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7fbc79800700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5c003a20 0x7fbc5c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f5ba5800700 1 --2- 172.21.15.3:0/3698503544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba00cda60 0x7f5ba00d0e30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009910 tx=0x7f5b94009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e85ee65dbbcaaa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f19600700 1 -- 172.21.15.3:0/885440098 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820408988 0 0) 0x7f8f1c06a930 con 0x7f8f1c06d580 2024-09-13T16:50:59.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f19600700 1 -- 172.21.15.3:0/885440098 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f000035d0 con 0x7f8f1c06d580 2024-09-13T16:50:59.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f19600700 1 -- 172.21.15.3:0/885440098 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607278127 0 0) 0x7f8f1c06ac10 con 0x7f8f1c1590d0 2024-09-13T16:50:59.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.557+0000 7f8f19600700 1 -- 172.21.15.3:0/885440098 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f1c06a930 con 0x7f8f1c1590d0 2024-09-13T16:50:59.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc78e00700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0043f0 0x7fbc5c0048d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc68005100 tx=0x7fbc5c0058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d576063d19527014 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1b400700 1 -- 172.21.15.3:0/885440098 >> v1:172.21.15.157:6789/0 conn(0x7f8f1c1590d0 legacy=0x7f8f1c06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1b400700 1 -- 172.21.15.3:0/885440098 >> v1:172.21.15.3:6789/0 conn(0x7f8f1c06d580 legacy=0x7f8f1c0b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1b400700 1 -- 172.21.15.3:0/885440098 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c06aef0 con 0x7f8f1c0cca20 2024-09-13T16:50:59.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5b9fe00700 1 -- 172.21.15.3:0/3698503544 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400ba60 con 0x7f5ba00cda60 2024-09-13T16:50:59.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5b9fe00700 1 -- 172.21.15.3:0/3698503544 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9400bbc0 con 0x7f5ba00cda60 2024-09-13T16:50:59.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5b9fe00700 1 -- 172.21.15.3:0/3698503544 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94009e60 con 0x7f5ba00cda60 2024-09-13T16:50:59.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecf400700 1 Processor -- start 2024-09-13T16:50:59.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecf400700 1 -- start start 2024-09-13T16:50:59.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68005aa0 con 0x7fbc5c0043f0 2024-09-13T16:50:59.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68005c00 con 0x7fbc5c0043f0 2024-09-13T16:50:59.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc73e00700 1 -- 172.21.15.3:0/1637011804 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680070d0 con 0x7fbc5c0043f0 2024-09-13T16:50:59.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc7c200700 1 -- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0043f0 msgr2=0x7fbc5c0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc7c200700 1 --2- 172.21.15.3:0/1637011804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0043f0 0x7fbc5c0048d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc68005100 tx=0x7fbc5c0058a0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5ba6c00700 1 -- 172.21.15.3:0/3698503544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba00cda60 msgr2=0x7f5ba00d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5ba6c00700 1 --2- 172.21.15.3:0/3698503544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba00cda60 0x7f5ba00d0e30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009910 tx=0x7f5b94009c20 comp rx=0 tx=0).stop 2024-09-13T16:50:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5ba6c00700 1 -- 172.21.15.3:0/3698503544 shutdown_connections 2024-09-13T16:50:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5ba6c00700 1 --2- 172.21.15.3:0/3698503544 >> v2:172.21.15.157:3300/0 conn(0x7f5ba00b8840 0x7f5ba00d2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5ba6c00700 1 --2- 172.21.15.3:0/3698503544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba00cda60 0x7f5ba00d0e30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5ba6c00700 1 -- 172.21.15.3:0/3698503544 >> 172.21.15.3:0/3698503544 conn(0x7f5ba00d31b0 msgr2=0x7f5ba00d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f47bf200700 1 -- 172.21.15.3:0/1056363333 shutdown_connections 2024-09-13T16:50:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f47bf200700 1 -- 172.21.15.3:0/1056363333 wait complete. 2024-09-13T16:50:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f1496000700 1 Processor -- start 2024-09-13T16:50:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc7a200700 1 -- 172.21.15.3:0/1637011804 reap_dead start 2024-09-13T16:50:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f1496000700 1 -- start start 2024-09-13T16:50:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc7c200700 1 -- 172.21.15.3:0/1637011804 shutdown_connections 2024-09-13T16:50:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc7c200700 1 -- 172.21.15.3:0/1637011804 >> 172.21.15.3:0/1637011804 conn(0x7fbc7406b1e0 msgr2=0x7fbc7406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec8072280 0x7f9ec8152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecf400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec80cc920 0x7f9ec80a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecf400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec81535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1b400700 1 --2- 172.21.15.3:0/885440098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c0cca20 0x7f8f1c0b2d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009dc0 tx=0x7f8f1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0870a8a01716256 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f19600700 1 -- 172.21.15.3:0/885440098 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400e070 con 0x7f8f1c0cca20 2024-09-13T16:50:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f19600700 1 -- 172.21.15.3:0/885440098 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1400bb20 con 0x7f8f1c0cca20 2024-09-13T16:50:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f19600700 1 -- 172.21.15.3:0/885440098 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14012430 con 0x7f8f1c0cca20 2024-09-13T16:50:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecf400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8156b50 con 0x7f9ec80cc920 2024-09-13T16:50:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc98015de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecf400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec81ee760 con 0x7f9ec8072280 2024-09-13T16:50:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecf400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8156870 con 0x7f9ec8157430 2024-09-13T16:50:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc98006b9f0 0x7fc9800b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc980156d40 0x7fc9801571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9800b34b0 con 0x7fc98006b9f0 2024-09-13T16:50:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98006d750 con 0x7fc980156d40 2024-09-13T16:50:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800b29b0 con 0x7fc98006b610 2024-09-13T16:50:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5ba6c00700 1 -- 172.21.15.3:0/3698503544 shutdown_connections 2024-09-13T16:50:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f5ba6c00700 1 -- 172.21.15.3:0/3698503544 wait complete. 2024-09-13T16:50:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7ff973400700 1 Processor -- start 2024-09-13T16:50:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ec7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec80cc920 0x7f9ec80a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ec7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec80cc920 0x7f9ec80a90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54572/0 (socket says 172.21.15.3:54572) 2024-09-13T16:50:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ec7e00700 1 -- 172.21.15.3:0/1255162071 learned_addr learned my addr 172.21.15.3:0/1255162071 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc7c200700 1 -- 172.21.15.3:0/1637011804 shutdown_connections 2024-09-13T16:50:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7ff973400700 1 -- start start 2024-09-13T16:50:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1be00700 1 -- 172.21.15.3:0/885440098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c0cca20 msgr2=0x7f8f1c0b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1be00700 1 --2- 172.21.15.3:0/885440098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c0cca20 0x7f8f1c0b2d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009dc0 tx=0x7f8f1400b330 comp rx=0 tx=0).stop 2024-09-13T16:50:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1be00700 1 -- 172.21.15.3:0/885440098 shutdown_connections 2024-09-13T16:50:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1be00700 1 --2- 172.21.15.3:0/885440098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c0cca20 0x7f8f1c0b2d50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f8f1be00700 1 -- 172.21.15.3:0/885440098 >> 172.21.15.3:0/885440098 conn(0x7f8f1c0b0cf0 msgr2=0x7f8f1c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc980156d40 0x7fc9801571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc980156d40 0x7fc9801571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:51962/0 (socket says 172.21.15.3:51962) 2024-09-13T16:50:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc985200700 1 -- 172.21.15.3:0/3153410221 learned_addr learned my addr 172.21.15.3:0/3153410221 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fc97fe00700 1 --2- 172.21.15.3:0/3153410221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc98015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc97f400700 1 --2- 172.21.15.3:0/3153410221 >> v2:172.21.15.3:3300/0 conn(0x7fc98006b9f0 0x7fc9800b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc985200700 1 -- 172.21.15.3:0/3153410221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 msgr2=0x7fc98015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc985200700 1 --2- 172.21.15.3:0/3153410221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc98015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc985200700 1 -- 172.21.15.3:0/3153410221 >> v2:172.21.15.3:3300/0 conn(0x7fc98006b9f0 msgr2=0x7fc9800b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc985200700 1 --2- 172.21.15.3:0/3153410221 >> v2:172.21.15.3:3300/0 conn(0x7fc98006b9f0 0x7fc9800b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47bf200700 1 Processor -- start 2024-09-13T16:50:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f8f1be00700 1 -- 172.21.15.3:0/885440098 shutdown_connections 2024-09-13T16:50:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc985200700 1 -- 172.21.15.3:0/3153410221 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98006d420 con 0x7fc980156d40 2024-09-13T16:50:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f8f1be00700 1 -- 172.21.15.3:0/885440098 wait complete. 2024-09-13T16:50:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc985200700 1 --2- 172.21.15.3:0/3153410221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980156d40 0x7fc9801571a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc9800af080 tx=0x7fc97800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1573cfbc7700c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff973400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff96c0b5b70 0x7ff96c0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47bf200700 1 -- start start 2024-09-13T16:50:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7fbc7c200700 1 -- 172.21.15.3:0/1637011804 wait complete. 2024-09-13T16:50:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fbc7c200700 1 Processor -- start 2024-09-13T16:50:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecd400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec81535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.561+0000 7f9ecca00700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec8072280 0x7f9ec8152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecd400700 1 -- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec8072280 msgr2=0x7f9ec8152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecd400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec8072280 0x7f9ec8152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecd400700 1 -- 172.21.15.3:0/1255162071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec80cc920 msgr2=0x7f9ec80a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecd400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec80cc920 0x7f9ec80a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecd400700 1 -- 172.21.15.3:0/1255162071 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0094b0 con 0x7f9ec8157430 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff973400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff96c06a390 0x7ff96c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff973400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c06ace0 con 0x7ff96c15b9f0 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff973400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c0b5120 con 0x7ff96c0b5b70 2024-09-13T16:50:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff973400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c15be80 con 0x7ff96c06a390 2024-09-13T16:50:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ec7e00700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec80cc920 0x7f9ec80a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecd400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec81535c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000cb90 tx=0x7f9ec000cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ec4c00700 1 -- 172.21.15.3:0/1255162071 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000a9e0 con 0x7f9ec8157430 2024-09-13T16:50:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ec4c00700 1 -- 172.21.15.3:0/1255162071 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec000ab40 con 0x7f9ec8157430 2024-09-13T16:50:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ec4c00700 1 -- 172.21.15.3:0/1255162071 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0007070 con 0x7f9ec8157430 2024-09-13T16:50:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec80a85d0 con 0x7f9ec8157430 2024-09-13T16:50:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec80a8b90 con 0x7f9ec8157430 2024-09-13T16:50:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f5ba6c00700 1 Processor -- start 2024-09-13T16:50:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff970a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff96c0b5b70 0x7ff96c0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff970a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff96c0b5b70 0x7ff96c0ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54592/0 (socket says 172.21.15.3:54592) 2024-09-13T16:50:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff970a00700 1 -- 172.21.15.3:0/870961230 learned_addr learned my addr 172.21.15.3:0/870961230 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc97ea00700 1 -- 172.21.15.3:0/3153410221 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800e070 con 0x7fc980156d40 2024-09-13T16:50:59.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc97ea00700 1 -- 172.21.15.3:0/3153410221 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97800b9c0 con 0x7fc980156d40 2024-09-13T16:50:59.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc97ea00700 1 -- 172.21.15.3:0/3153410221 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800bd50 con 0x7fc980156d40 2024-09-13T16:50:59.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff971400700 1 --2- 172.21.15.3:0/870961230 >> v2:172.21.15.157:3300/0 conn(0x7ff96c06a390 0x7ff96c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff970a00700 1 -- 172.21.15.3:0/870961230 >> v2:172.21.15.157:3300/0 conn(0x7ff96c06a390 msgr2=0x7ff96c06a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff970a00700 1 --2- 172.21.15.3:0/870961230 >> v2:172.21.15.157:3300/0 conn(0x7ff96c06a390 0x7ff96c06a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff970a00700 1 -- 172.21.15.3:0/870961230 >> v1:172.21.15.157:6789/0 conn(0x7ff96c15b9f0 legacy=0x7ff96c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7ff970a00700 1 -- 172.21.15.3:0/870961230 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0cd5c0 con 0x7ff96c0b5b70 2024-09-13T16:50:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f1496000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f149006d550 0x7f14900717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f1496000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900b2c90 con 0x7f14900ce3b0 2024-09-13T16:50:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f1496000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900ce790 con 0x7f149006d0e0 2024-09-13T16:50:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f1496000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14900b2080 con 0x7f149006d550 2024-09-13T16:50:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47bf200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 0x7f47c0159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47bf200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c00b1bd0 0x7f47c01585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47bf200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c0142030 0x7f47c01386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47bf200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c0159950 con 0x7f47c006cdd0 2024-09-13T16:50:59.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47bf200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c0159c30 con 0x7f47c0142030 2024-09-13T16:50:59.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47bf200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c0159f10 con 0x7f47c00b1bd0 2024-09-13T16:50:59.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47be800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c0142030 0x7f47c01386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47be800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c0142030 0x7f47c01386a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:51982/0 (socket says 172.21.15.3:51982) 2024-09-13T16:50:59.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f47be800700 1 -- 172.21.15.3:0/1203421713 learned_addr learned my addr 172.21.15.3:0/1203421713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec80b4ba0 con 0x7f9ec8157430 2024-09-13T16:50:59.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7fc985c00700 1 -- 172.21.15.3:0/3153410221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980156d40 msgr2=0x7fc9801571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fc985c00700 1 --2- 172.21.15.3:0/3153410221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980156d40 0x7fc9801571a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc9800af080 tx=0x7fc97800b330 comp rx=0 tx=0).stop 2024-09-13T16:50:59.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bd400700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c00b1bd0 0x7f47c01585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148ea00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f149006d0e0 0x7f14900710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37564/0 (socket says 172.21.15.3:37564) 2024-09-13T16:50:59.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148ea00700 1 -- 172.21.15.3:0/3803269372 learned_addr learned my addr 172.21.15.3:0/3803269372 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:50:59.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148fe00700 1 --2- 172.21.15.3:0/3803269372 >> v2:172.21.15.3:3300/0 conn(0x7f149006d550 0x7f14900717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fc985c00700 1 -- 172.21.15.3:0/3153410221 shutdown_connections 2024-09-13T16:50:59.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fc985c00700 1 --2- 172.21.15.3:0/3153410221 >> v2:172.21.15.3:3300/0 conn(0x7fc98006b9f0 0x7fc9800b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fc985c00700 1 --2- 172.21.15.3:0/3153410221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980156d40 0x7fc9801571a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fc985c00700 1 --2- 172.21.15.3:0/3153410221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc98015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fc985c00700 1 -- 172.21.15.3:0/3153410221 >> 172.21.15.3:0/3153410221 conn(0x7fc9800706d0 msgr2=0x7fc980070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bde00700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 0x7f47c0159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fbc7c200700 1 -- start start 2024-09-13T16:50:59.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125919546 0 0) 0x7f14900ce790 con 0x7f149006d0e0 2024-09-13T16:50:59.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14740035d0 con 0x7f149006d0e0 2024-09-13T16:50:59.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7ff970a00700 1 --2- 172.21.15.3:0/870961230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 0x7ff96c0ccec0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0cc400 tx=0x7ff95c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b1a551c2f5d84e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bd400700 1 -- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c0142030 msgr2=0x7f47c01386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bd400700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c0142030 0x7f47c01386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bd400700 1 -- 172.21.15.3:0/1203421713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 msgr2=0x7f47c0159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bd400700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 0x7f47c0159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bd400700 1 -- 172.21.15.3:0/1203421713 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b000a650 con 0x7f47c00b1bd0 2024-09-13T16:50:59.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2402637553 0 0) 0x7f14740035d0 con 0x7f149006d0e0 2024-09-13T16:50:59.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 >> v2:172.21.15.3:3300/0 conn(0x7f149006d550 msgr2=0x7f14900717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 --2- 172.21.15.3:0/3803269372 >> v2:172.21.15.3:3300/0 conn(0x7f149006d550 0x7f14900717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 >> v1:172.21.15.3:6789/0 conn(0x7f14900ce3b0 legacy=0x7f149015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900b2360 con 0x7f149006d0e0 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f147c001670 con 0x7f149006d0e0 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 >> v1:172.21.15.157:6789/0 conn(0x7f149006d0e0 legacy=0x7f14900710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1474003a20 0x7f1474003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1474004870 0x7f1474004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474005230 0x7f147400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b2080 con 0x7f1474003a20 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147400c180 con 0x7f1474004870 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147400c340 con 0x7f1474005230 2024-09-13T16:50:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bd400700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c00b1bd0 0x7f47c01585f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b0015000 tx=0x7f47b0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.565+0000 7f5ba6c00700 1 -- start start 2024-09-13T16:50:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba6c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba00b8840 0x7f5ba01fedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba6c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ba00cda60 0x7f5ba00c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba6c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba015d220 0x7f5ba00c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba6c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00ca240 con 0x7f5ba00b8840 2024-09-13T16:50:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba6c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba0155ec0 con 0x7f5ba00cda60 2024-09-13T16:50:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba6c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba0156ee0 con 0x7f5ba015d220 2024-09-13T16:50:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fc985c00700 1 -- 172.21.15.3:0/3153410221 shutdown_connections 2024-09-13T16:50:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148ea00700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1474004870 0x7f1474004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148f400700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1474003a20 0x7f1474003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148fe00700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474005230 0x7f147400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7ff96b400700 1 -- 172.21.15.3:0/870961230 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004230 con 0x7ff96c0b5b70 2024-09-13T16:50:59.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7ff96b400700 1 -- 172.21.15.3:0/870961230 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c004390 con 0x7ff96c0b5b70 2024-09-13T16:50:59.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7ff96b400700 1 -- 172.21.15.3:0/870961230 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004230 con 0x7ff96c0b5b70 2024-09-13T16:50:59.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47b6000700 1 -- 172.21.15.3:0/1203421713 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b001d070 con 0x7f47c00b1bd0 2024-09-13T16:50:59.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c015a1f0 con 0x7f47c00b1bd0 2024-09-13T16:50:59.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148ea00700 1 -- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474005230 msgr2=0x7f147400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148ea00700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474005230 0x7f147400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148ea00700 1 -- 172.21.15.3:0/3803269372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1474003a20 msgr2=0x7f1474003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148ea00700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1474003a20 0x7f1474003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148ea00700 1 -- 172.21.15.3:0/3803269372 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14740035d0 con 0x7f1474004870 2024-09-13T16:50:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f148fe00700 1 -- 172.21.15.3:0/3803269372 reap_dead start 2024-09-13T16:50:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f8f1be00700 1 Processor -- start 2024-09-13T16:50:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba4e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ba00cda60 0x7f5ba00c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba4e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ba00cda60 0x7f5ba00c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52002/0 (socket says 172.21.15.3:52002) 2024-09-13T16:50:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba4e00700 1 -- 172.21.15.3:0/561815105 learned_addr learned my addr 172.21.15.3:0/561815105 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c0138be0 con 0x7f47c00b1bd0 2024-09-13T16:50:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47b6000700 1 -- 172.21.15.3:0/1203421713 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b00038d0 con 0x7f47c00b1bd0 2024-09-13T16:50:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f47b6000700 1 -- 172.21.15.3:0/1203421713 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0004230 con 0x7f47c00b1bd0 2024-09-13T16:50:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba6200700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba015d220 0x7f5ba00c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba5800700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba00b8840 0x7f5ba01fedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba4e00700 1 -- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba015d220 msgr2=0x7f5ba00c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7ff973400700 1 -- 172.21.15.3:0/870961230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 msgr2=0x7ff96c0ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7ff973400700 1 --2- 172.21.15.3:0/870961230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 0x7ff96c0ccec0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0cc400 tx=0x7ff95c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba4e00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba015d220 0x7f5ba00c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba4e00700 1 -- 172.21.15.3:0/561815105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba00b8840 msgr2=0x7f5ba01fedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba4e00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba00b8840 0x7f5ba01fedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f5ba4e00700 1 -- 172.21.15.3:0/561815105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94009570 con 0x7f5ba00cda60 2024-09-13T16:50:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7ff973400700 1 -- 172.21.15.3:0/870961230 shutdown_connections 2024-09-13T16:50:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7ff973400700 1 --2- 172.21.15.3:0/870961230 >> v2:172.21.15.157:3300/0 conn(0x7ff96c06a390 0x7ff96c06a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7ff973400700 1 --2- 172.21.15.3:0/870961230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 0x7ff96c0ccec0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7ff973400700 1 -- 172.21.15.3:0/870961230 >> 172.21.15.3:0/870961230 conn(0x7ff96c071d80 msgr2=0x7ff96c06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f9ec4c00700 1 -- 172.21.15.3:0/1255162071 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ec00071d0 con 0x7f9ec8157430 2024-09-13T16:50:59.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc7c200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc741365f0 0x7fbc74131c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f5ba4e00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ba00cda60 0x7f5ba00c7eb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00c440 tx=0x7f5b8c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7fc985c00700 1 -- 172.21.15.3:0/3153410221 wait complete. 2024-09-13T16:50:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7ff973400700 1 -- 172.21.15.3:0/870961230 shutdown_connections 2024-09-13T16:50:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985c00700 1 Processor -- start 2024-09-13T16:50:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f148ea00700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1474004870 0x7f1474004cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f147c00c320 tx=0x7f14740058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d728abc681aba811 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc7c200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc741369d0 0x7fbc7415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985c00700 1 -- start start 2024-09-13T16:50:59.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.569+0000 7f8f1be00700 1 -- start start 2024-09-13T16:50:59.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4798005310 con 0x7f47c00b1bd0 2024-09-13T16:50:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c06d580 0x7f8f1c1ff140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f1c0cca20 0x7f8f1c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1590d0 0x7f8f1c14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1be00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c150fc0 con 0x7f8f1c0cca20 2024-09-13T16:50:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c14d240 con 0x7f8f1c1590d0 2024-09-13T16:50:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1be00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c150ce0 con 0x7f8f1c06d580 2024-09-13T16:50:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c007c30 con 0x7f1474004870 2024-09-13T16:50:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7ff973400700 1 -- 172.21.15.3:0/870961230 wait complete. 2024-09-13T16:50:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c00ce80 con 0x7f1474004870 2024-09-13T16:50:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f148e000700 1 -- 172.21.15.3:0/3803269372 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c005bc0 con 0x7f1474004870 2024-09-13T16:50:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f1496000700 1 -- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1474004870 msgr2=0x7f1474004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f1496000700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1474004870 0x7f1474004cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f147c00c320 tx=0x7f14740058a0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f5b9d600700 1 -- 172.21.15.3:0/561815105 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0052d0 con 0x7f5ba00cda60 2024-09-13T16:50:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f5b9d600700 1 -- 172.21.15.3:0/561815105 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c005430 con 0x7f5ba00cda60 2024-09-13T16:50:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f5b9d600700 1 -- 172.21.15.3:0/561815105 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0055a0 con 0x7f5ba00cda60 2024-09-13T16:50:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f5ba6c00700 1 -- 172.21.15.3:0/561815105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba00d2c00 con 0x7f5ba00cda60 2024-09-13T16:50:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f5ba6c00700 1 -- 172.21.15.3:0/561815105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba01574a0 con 0x7f5ba00cda60 2024-09-13T16:50:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1590d0 0x7f8f1c14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1aa00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c06d580 0x7f8f1c1ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1590d0 0x7f8f1c14e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52016/0 (socket says 172.21.15.3:52016) 2024-09-13T16:50:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f9ec4c00700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ea8077740 0x7f9ea8079c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1b400700 1 -- 172.21.15.3:0/2147612211 learned_addr learned my addr 172.21.15.3:0/2147612211 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc980142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98006b9f0 0x7fc980160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc980156d40 0x7fc980161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98015a080 con 0x7fc980156d40 2024-09-13T16:50:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801597c0 con 0x7fc98006b9f0 2024-09-13T16:50:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc980143220 con 0x7fc98006b610 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc980142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc980142ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52742/0 (socket says 172.21.15.3:52742) 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97fe00700 1 -- 172.21.15.3:0/2166871992 learned_addr learned my addr 172.21.15.3:0/2166871992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1a000700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f1c0cca20 0x7f8f1c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1aa00700 1 -- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1590d0 msgr2=0x7f8f1c14e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1aa00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1590d0 0x7f8f1c14e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1aa00700 1 -- 172.21.15.3:0/2147612211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f1c0cca20 msgr2=0x7f8f1c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1aa00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f1c0cca20 0x7f8f1c158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1aa00700 1 -- 172.21.15.3:0/2147612211 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f14009a70 con 0x7f8f1c06d580 2024-09-13T16:50:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f9ec4c00700 1 -- 172.21.15.3:0/1255162071 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ec00845a0 con 0x7f9ec8157430 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f9ecca00700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ea8077740 0x7f9ea8079c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc7c200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc741312b0 0x7fbc741fe830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc7a200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc741312b0 0x7fbc741fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc79800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc741365f0 0x7fbc74131c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc79800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc741365f0 0x7fbc74131c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52006/0 (socket says 172.21.15.3:52006) 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc79800700 1 -- 172.21.15.3:0/187436546 learned_addr learned my addr 172.21.15.3:0/187436546 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc78e00700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc741369d0 0x7fbc7415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7415d4f0 con 0x7fbc741312b0 2024-09-13T16:50:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74131a90 con 0x7fbc741365f0 2024-09-13T16:50:59.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74132310 con 0x7fbc741369d0 2024-09-13T16:50:59.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97f400700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98006b9f0 0x7fc980160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1b400700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1590d0 0x7f8f1c14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:59.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc985200700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc980156d40 0x7fc980161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f5ba6c00700 1 -- 172.21.15.3:0/561815105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba0067730 con 0x7f5ba00cda60 2024-09-13T16:50:59.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7ff973400700 1 Processor -- start 2024-09-13T16:50:59.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7f8f1aa00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c06d580 0x7f8f1c1ff140 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c410 tx=0x7f8f0c00c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97fe00700 1 -- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98006b9f0 msgr2=0x7fc980160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97fe00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98006b9f0 0x7fc980160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97fe00700 1 -- 172.21.15.3:0/2166871992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc980156d40 msgr2=0x7fc980161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97fe00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc980156d40 0x7fc980161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7fc97fe00700 1 -- 172.21.15.3:0/2166871992 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc978009b30 con 0x7fc98006b610 2024-09-13T16:50:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.573+0000 7ff973400700 1 -- start start 2024-09-13T16:50:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 -- 172.21.15.3:0/3803269372 shutdown_connections 2024-09-13T16:50:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 --2- 172.21.15.3:0/3803269372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1474004870 0x7f1474004cf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 -- 172.21.15.3:0/3803269372 >> 172.21.15.3:0/3803269372 conn(0x7f149006a390 msgr2=0x7f149006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7fbc78e00700 1 -- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc741365f0 msgr2=0x7fbc74131c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7fbc78e00700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc741365f0 0x7fbc74131c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7fbc78e00700 1 -- 172.21.15.3:0/187436546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc741312b0 msgr2=0x7fbc741fe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7fbc78e00700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc741312b0 0x7fbc741fe830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7fbc78e00700 1 -- 172.21.15.3:0/187436546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc741325f0 con 0x7fbc741369d0 2024-09-13T16:50:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7fc97fe00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc980142ce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc97000c820 tx=0x7fc97000cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 -- 172.21.15.3:0/3803269372 shutdown_connections 2024-09-13T16:50:59.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 -- 172.21.15.3:0/3803269372 wait complete. 2024-09-13T16:50:59.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7fbc78e00700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc741369d0 0x7fbc7415cfb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc680050d0 tx=0x7fbc68004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f5b9d600700 1 -- 172.21.15.3:0/561815105 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b8c0038e0 con 0x7f5ba00cda60 2024-09-13T16:50:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f9ecca00700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ea8077740 0x7f9ea8079c00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0090b0 tx=0x7f9ebc012000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff973400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c06a390 0x7ff96c149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff973400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 0x7ff96c14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff973400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff96c15b9f0 0x7ff96c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff973400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c14bdb0 con 0x7ff96c0b5b70 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff973400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c14b130 con 0x7ff96c06a390 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff973400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c14b410 con 0x7ff96c15b9f0 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c06a390 0x7ff96c149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c06a390 0x7ff96c149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52028/0 (socket says 172.21.15.3:52028) 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 -- 172.21.15.3:0/1029616633 learned_addr learned my addr 172.21.15.3:0/1029616633 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff96be00700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 0x7ff96c14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff971400700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff96c15b9f0 0x7ff96c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 -- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff96c15b9f0 msgr2=0x7ff96c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff96c15b9f0 0x7ff96c15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 -- 172.21.15.3:0/1029616633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 msgr2=0x7ff96c14a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 0x7ff96c14a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 -- 172.21.15.3:0/1029616633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c009570 con 0x7ff96c06a390 2024-09-13T16:50:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f5b9d600700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5b80066780 0x7f5b80068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f5b9d600700 1 -- 172.21.15.3:0/561815105 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5b8c083700 con 0x7f5ba00cda60 2024-09-13T16:50:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f5ba5800700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5b80066780 0x7f5b80068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7ff970a00700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c06a390 0x7ff96c149cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff96c067810 tx=0x7ff95c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 Processor -- start 2024-09-13T16:50:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 -- start start 2024-09-13T16:50:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f5ba5800700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5b80066780 0x7f5b80068c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009c60 tx=0x7f5b94008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474004870 0x7f14901464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1490153410 0x7f1490153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.577+0000 7f1496000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f149015c020 0x7f149015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f1496000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149015c9e0 con 0x7f149015c020 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f1496000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14901540a0 con 0x7f1490153410 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f1496000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1490154380 con 0x7f1474004870 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f149015c020 0x7f149015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f149015c020 0x7f149015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54668/0 (socket says 172.21.15.3:54668) 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 -- 172.21.15.3:0/2485278144 learned_addr learned my addr 172.21.15.3:0/2485278144 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148ea00700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1490153410 0x7f1490153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148f400700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474004870 0x7f14901464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 -- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474004870 msgr2=0x7f14901464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474004870 0x7f14901464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 -- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1490153410 msgr2=0x7f1490153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1490153410 0x7f1490153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 -- 172.21.15.3:0/2485278144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1490154660 con 0x7f149015c020 2024-09-13T16:50:59.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148ea00700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1490153410 0x7f1490153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:59.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f148fe00700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f149015c020 0x7f149015c4a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14900b4160 tx=0x7f14880048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f9ec4c00700 1 -- 172.21.15.3:0/1255162071 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ec0051690 con 0x7f9ec8157430 2024-09-13T16:50:59.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fbc71600700 1 -- 172.21.15.3:0/187436546 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68007730 con 0x7fbc741369d0 2024-09-13T16:50:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fc977e00700 1 -- 172.21.15.3:0/2166871992 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970011800 con 0x7fc98006b610 2024-09-13T16:50:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fbc71600700 1 -- 172.21.15.3:0/187436546 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6801d630 con 0x7fbc741369d0 2024-09-13T16:50:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc741328d0 con 0x7fbc741369d0 2024-09-13T16:50:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fbc71600700 1 -- 172.21.15.3:0/187436546 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6801d9e0 con 0x7fbc741369d0 2024-09-13T16:50:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fc977e00700 1 -- 172.21.15.3:0/2166871992 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc970011960 con 0x7fc98006b610 2024-09-13T16:50:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc74132e90 con 0x7fbc741369d0 2024-09-13T16:50:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f1487400700 1 -- 172.21.15.3:0/2485278144 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488007960 con 0x7f149015c020 2024-09-13T16:50:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f1487400700 1 -- 172.21.15.3:0/2485278144 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1488007ac0 con 0x7f149015c020 2024-09-13T16:50:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f1487400700 1 -- 172.21.15.3:0/2485278144 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488007e70 con 0x7f149015c020 2024-09-13T16:50:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1490154940 con 0x7f149015c020 2024-09-13T16:50:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f8f12a00700 1 -- 172.21.15.3:0/2147612211 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c005d90 con 0x7f8f1c06d580 2024-09-13T16:50:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f8f12a00700 1 -- 172.21.15.3:0/2147612211 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0070d0 con 0x7f8f1c06d580 2024-09-13T16:50:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f8f12a00700 1 -- 172.21.15.3:0/2147612211 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c007440 con 0x7f8f1c06d580 2024-09-13T16:50:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fc977e00700 1 -- 172.21.15.3:0/2166871992 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970011cf0 con 0x7fc98006b610 2024-09-13T16:50:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc980159a30 con 0x7fc98006b610 2024-09-13T16:50:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9801f6850 con 0x7fc98006b610 2024-09-13T16:50:59.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1490154f00 con 0x7f149015c020 2024-09-13T16:50:59.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c14cbf0 con 0x7f8f1c06d580 2024-09-13T16:50:59.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c1ff680 con 0x7f8f1c06d580 2024-09-13T16:50:59.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.581+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc74070070 con 0x7fbc741369d0 2024-09-13T16:50:59.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f47b6000700 1 -- 172.21.15.3:0/1203421713 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47b0003a60 con 0x7f47c00b1bd0 2024-09-13T16:50:59.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9800b2470 con 0x7fc98006b610 2024-09-13T16:50:59.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7ff968c00700 1 -- 172.21.15.3:0/1029616633 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00b660 con 0x7ff96c06a390 2024-09-13T16:50:59.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7ff968c00700 1 -- 172.21.15.3:0/1029616633 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c00b7e0 con 0x7ff96c06a390 2024-09-13T16:50:59.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7ff968c00700 1 -- 172.21.15.3:0/1029616633 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00bb90 con 0x7ff96c06a390 2024-09-13T16:50:59.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14901590c0 con 0x7f149015c020 2024-09-13T16:50:59.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1c14ca50 con 0x7f8f1c06d580 2024-09-13T16:50:59.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f47b6000700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f479c066830 0x7f479c068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f47b6000700 1 -- 172.21.15.3:0/1203421713 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f47b0085790 con 0x7f47c00b1bd0 2024-09-13T16:50:59.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c14b6f0 con 0x7ff96c06a390 2024-09-13T16:50:59.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c14a1f0 con 0x7ff96c06a390 2024-09-13T16:50:59.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f47bde00700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f479c066830 0x7f479c068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f5b9d600700 1 -- 172.21.15.3:0/561815105 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b8c0509b0 con 0x7f5ba00cda60 2024-09-13T16:50:59.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f47b6000700 1 -- 172.21.15.3:0/1203421713 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f47b00529a0 con 0x7f47c00b1bd0 2024-09-13T16:50:59.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f47bde00700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f479c066830 0x7f479c068cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f47b80012e0 tx=0x7f47b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7f1487400700 1 -- 172.21.15.3:0/2485278144 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1488015420 con 0x7f149015c020 2024-09-13T16:50:59.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.589+0000 7f1487400700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f14700667b0 0x7f1470068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.589+0000 7f1487400700 1 -- 172.21.15.3:0/2485278144 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f14880847f0 con 0x7f149015c020 2024-09-13T16:50:59.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.589+0000 7f148f400700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f14700667b0 0x7f1470068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.585+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c14bef0 con 0x7ff96c06a390 2024-09-13T16:50:59.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.589+0000 7ff968c00700 1 -- 172.21.15.3:0/1029616633 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff95c00bcf0 con 0x7ff96c06a390 2024-09-13T16:50:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.589+0000 7fbc71600700 1 -- 172.21.15.3:0/187436546 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc6801db40 con 0x7fbc741369d0 2024-09-13T16:50:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7ff968c00700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff94c066800 0x7ff94c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.589+0000 7fc977e00700 1 -- 172.21.15.3:0/2166871992 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc970024480 con 0x7fc98006b610 2024-09-13T16:50:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7ff968c00700 1 -- 172.21.15.3:0/1029616633 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff95c085c50 con 0x7ff96c06a390 2024-09-13T16:50:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.589+0000 7f8f12a00700 1 -- 172.21.15.3:0/2147612211 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8f0c0075a0 con 0x7f8f1c06d580 2024-09-13T16:50:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7f8f12a00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8efc066800 0x7f8efc068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7fc977e00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc960066800 0x7fc960068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7fc977e00700 1 -- 172.21.15.3:0/2166871992 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc970084770 con 0x7fc98006b610 2024-09-13T16:50:59.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7f8f12a00700 1 -- 172.21.15.3:0/2147612211 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8f0c083aa0 con 0x7f8f1c06d580 2024-09-13T16:50:59.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7ff96be00700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff94c066800 0x7ff94c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7f148f400700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f14700667b0 0x7f1470068c70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7f8f1a000700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8efc066800 0x7f8efc068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7fc97f400700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc960066800 0x7fc960068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7fc97f400700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc960066800 0x7fc960068cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc98015e120 tx=0x7fc96c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7ff96be00700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff94c066800 0x7ff94c068cc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff96c158790 tx=0x7ff964009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7f8f1a000700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8efc066800 0x7f8efc068cc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8f08007fd0 tx=0x7f8f08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7fbc71600700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbc54066760 0x7fbc54068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7fbc71600700 1 -- 172.21.15.3:0/187436546 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbc680855b0 con 0x7fbc741369d0 2024-09-13T16:50:59.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7fbc79800700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbc54066760 0x7fbc54068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.593+0000 7fbc71600700 1 -- 172.21.15.3:0/187436546 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbc680526a0 con 0x7fbc741369d0 2024-09-13T16:50:59.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.597+0000 7ff968c00700 1 -- 172.21.15.3:0/1029616633 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff96c14bef0 con 0x7ff96c06a390 2024-09-13T16:50:59.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.597+0000 7fbc79800700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbc54066760 0x7fbc54068c20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc600017a0 tx=0x7fbc60007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.597+0000 7f8f12a00700 1 -- 172.21.15.3:0/2147612211 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8f0c050bb0 con 0x7f8f1c06d580 2024-09-13T16:50:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.597+0000 7f1487400700 1 -- 172.21.15.3:0/2485278144 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f14880518e0 con 0x7f149015c020 2024-09-13T16:50:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.609+0000 7fc977e00700 1 -- 172.21.15.3:0/2166871992 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc9700517d0 con 0x7fc98006b610 2024-09-13T16:50:59.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.933+0000 7f5ba6c00700 1 -- 172.21.15.3:0/561815105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5ba01560c0 con 0x7f5ba00cda60 2024-09-13T16:50:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.933+0000 7f5b9d600700 1 -- 172.21.15.3:0/561815105 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b8c053fd0 con 0x7f5ba00cda60 2024-09-13T16:50:59.941 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2024-09-13T16:50:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 -- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5b80066780 msgr2=0x7f5b80068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5b80066780 0x7f5b80068c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009c60 tx=0x7f5b94008030 comp rx=0 tx=0).stop 2024-09-13T16:50:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 -- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ba00cda60 msgr2=0x7f5ba00c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ba00cda60 0x7f5ba00c7eb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00c440 tx=0x7f5b8c001e70 comp rx=0 tx=0).stop 2024-09-13T16:50:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 -- 172.21.15.3:0/561815105 shutdown_connections 2024-09-13T16:50:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5b80066780 0x7f5b80068c40 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009c60 tx=0x7f5b94008030 comp rx=0 tx=0).stop 2024-09-13T16:50:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ba015d220 0x7f5ba00c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ba00cda60 0x7f5ba00c7eb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 --2- 172.21.15.3:0/561815105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba00b8840 0x7f5ba01fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 -- 172.21.15.3:0/561815105 >> 172.21.15.3:0/561815105 conn(0x7f5ba00d31b0 msgr2=0x7f5ba00b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 -- 172.21.15.3:0/561815105 shutdown_connections 2024-09-13T16:50:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.937+0000 7f5b92a00700 1 -- 172.21.15.3:0/561815105 wait complete. 2024-09-13T16:50:59.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.945+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9ec8157810 con 0x7f9ec8157430 2024-09-13T16:50:59.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.949+0000 7f9ec4c00700 1 -- 172.21.15.3:0/1255162071 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ec0017020 con 0x7f9ec8157430 2024-09-13T16:50:59.952 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:50:59.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.949+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f14900b3510 con 0x7f149015c020 2024-09-13T16:50:59.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.949+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ea8077740 msgr2=0x7f9ea8079c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.949+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbc74138400 con 0x7fbc741369d0 2024-09-13T16:50:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ea8077740 0x7f9ea8079c00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0090b0 tx=0x7f9ebc012000 comp rx=0 tx=0).stop 2024-09-13T16:50:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 msgr2=0x7f9ec81535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec81535c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000cb90 tx=0x7f9ec000cea0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 shutdown_connections 2024-09-13T16:50:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ea8077740 0x7f9ea8079c00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f1487400700 1 -- 172.21.15.3:0/2485278144 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1488054f00 con 0x7f149015c020 2024-09-13T16:50:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec8157430 0x7f9ec81535c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7fbc71600700 1 -- 172.21.15.3:0/187436546 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbc68017020 con 0x7fbc741369d0 2024-09-13T16:50:59.958 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:50:59.958 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:50:59.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec80cc920 0x7f9ec80a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 --2- 172.21.15.3:0/1255162071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec8072280 0x7f9ec8152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff96c14bef0 con 0x7ff96c06a390 2024-09-13T16:50:59.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 >> 172.21.15.3:0/1255162071 conn(0x7f9ec8071050 msgr2=0x7f9ec806ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 shutdown_connections 2024-09-13T16:50:59.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f9ecf400700 1 -- 172.21.15.3:0/1255162071 wait complete. 2024-09-13T16:50:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4798005890 con 0x7f47c00b1bd0 2024-09-13T16:50:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7ff968c00700 1 -- 172.21.15.3:0/1029616633 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff96c14bef0 con 0x7ff96c06a390 2024-09-13T16:50:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.953+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8f1c14ca50 con 0x7f8f1c06d580 2024-09-13T16:50:59.961 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:50:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f14700667b0 msgr2=0x7f1470068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f14700667b0 0x7f1470068c70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480009040 comp rx=0 tx=0).stop 2024-09-13T16:50:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f149015c020 msgr2=0x7f149015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f149015c020 0x7f149015c4a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14900b4160 tx=0x7f14880048c0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 shutdown_connections 2024-09-13T16:50:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f14700667b0 0x7f1470068c70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f149015c020 0x7f149015c4a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1490153410 0x7f1490153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 --2- 172.21.15.3:0/2485278144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1474004870 0x7f14901464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 >> 172.21.15.3:0/2485278144 conn(0x7f149006a390 msgr2=0x7f149015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f8f12a00700 1 -- 172.21.15.3:0/2147612211 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f0c017020 con 0x7f8f1c06d580 2024-09-13T16:50:59.964 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2024-09-13T16:50:59.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f47b6000700 1 -- 172.21.15.3:0/1203421713 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47b0026020 con 0x7f47c00b1bd0 2024-09-13T16:50:59.964 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:50:59.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbc54066760 msgr2=0x7fbc54068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbc54066760 0x7fbc54068c20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc600017a0 tx=0x7fbc60007040 comp rx=0 tx=0).stop 2024-09-13T16:50:59.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc741369d0 msgr2=0x7fbc7415cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc741369d0 0x7fbc7415cfb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc680050d0 tx=0x7fbc68004970 comp rx=0 tx=0).stop 2024-09-13T16:50:59.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 shutdown_connections 2024-09-13T16:50:59.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7f1496000700 1 -- 172.21.15.3:0/2485278144 wait complete. 2024-09-13T16:50:59.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 shutdown_connections 2024-09-13T16:50:59.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbc54066760 0x7fbc54068c20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc741312b0 0x7fbc741fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc741369d0 0x7fbc7415cfb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 --2- 172.21.15.3:0/187436546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc741365f0 0x7fbc74131c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.957+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 >> 172.21.15.3:0/187436546 conn(0x7fbc7406b1e0 msgr2=0x7fbc74137650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 shutdown_connections 2024-09-13T16:50:59.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7fbc7c200700 1 -- 172.21.15.3:0/187436546 wait complete. 2024-09-13T16:50:59.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff94c066800 msgr2=0x7ff94c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff94c066800 0x7ff94c068cc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff96c158790 tx=0x7ff964009040 comp rx=0 tx=0).stop 2024-09-13T16:50:59.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c06a390 msgr2=0x7ff96c149cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c06a390 0x7ff96c149cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff96c067810 tx=0x7ff95c004770 comp rx=0 tx=0).stop 2024-09-13T16:50:59.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f479c066830 msgr2=0x7f479c068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f479c066830 0x7f479c068cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f47b80012e0 tx=0x7f47b8009040 comp rx=0 tx=0).stop 2024-09-13T16:50:59.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c00b1bd0 msgr2=0x7f47c01585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c00b1bd0 0x7f47c01585f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b0015000 tx=0x7f47b0004770 comp rx=0 tx=0).stop 2024-09-13T16:50:59.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8efc066800 msgr2=0x7f8efc068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8efc066800 0x7f8efc068cc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8f08007fd0 tx=0x7f8f08009040 comp rx=0 tx=0).stop 2024-09-13T16:50:59.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c06d580 msgr2=0x7f8f1c1ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c06d580 0x7f8f1c1ff140 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c410 tx=0x7f8f0c00c720 comp rx=0 tx=0).stop 2024-09-13T16:50:59.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 shutdown_connections 2024-09-13T16:50:59.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8efc066800 0x7f8efc068cc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1590d0 0x7f8f1c14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f1c0cca20 0x7f8f1c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 --2- 172.21.15.3:0/2147612211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1c06d580 0x7f8f1c1ff140 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 >> 172.21.15.3:0/2147612211 conn(0x7f8f1c0b0cf0 msgr2=0x7f8f1c070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 shutdown_connections 2024-09-13T16:50:59.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f8f1be00700 1 -- 172.21.15.3:0/2147612211 wait complete. 2024-09-13T16:50:59.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 shutdown_connections 2024-09-13T16:50:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f479c066830 0x7f479c068cf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c0142030 0x7f47c01386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c00b1bd0 0x7f47c01585f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 --2- 172.21.15.3:0/1203421713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47c006cdd0 0x7f47c0159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 >> 172.21.15.3:0/1203421713 conn(0x7f47c00b0cf0 msgr2=0x7f47c0070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 shutdown_connections 2024-09-13T16:50:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff94c066800 0x7ff94c068cc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff96c15b9f0 0x7ff96c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0b5b70 0x7ff96c14a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 --2- 172.21.15.3:0/1029616633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c06a390 0x7ff96c149cb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 >> 172.21.15.3:0/1029616633 conn(0x7ff96c071d80 msgr2=0x7ff96c0b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 shutdown_connections 2024-09-13T16:50:59.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7f47bf200700 1 -- 172.21.15.3:0/1203421713 wait complete. 2024-09-13T16:50:59.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 shutdown_connections 2024-09-13T16:50:59.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.7 2024-09-13T16:50:59.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.961+0000 7ff973400700 1 -- 172.21.15.3:0/1029616633 wait complete. 2024-09-13T16:50:59.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.965+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc980156170 con 0x7fc98006b610 2024-09-13T16:50:59.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.965+0000 7fc977e00700 1 -- 172.21.15.3:0/2166871992 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc97001f090 con 0x7fc98006b610 2024-09-13T16:50:59.975 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:50:59.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.969+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc960066800 msgr2=0x7fc960068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.969+0000 7fc985c00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc960066800 0x7fc960068cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc98015e120 tx=0x7fc96c008040 comp rx=0 tx=0).stop 2024-09-13T16:50:59.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.969+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 msgr2=0x7fc980142ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.969+0000 7fc985c00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc980142ce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc97000c820 tx=0x7fc97000cb30 comp rx=0 tx=0).stop 2024-09-13T16:50:59.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.973+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 shutdown_connections 2024-09-13T16:50:59.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.973+0000 7fc985c00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc960066800 0x7fc960068cc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.973+0000 7fc985c00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc980156d40 0x7fc980161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.973+0000 7fc985c00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98006b9f0 0x7fc980160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.973+0000 7fc985c00700 1 --2- 172.21.15.3:0/2166871992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98006b610 0x7fc980142ce0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.973+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 >> 172.21.15.3:0/2166871992 conn(0x7fc9800706d0 msgr2=0x7fc9800ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.973+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 shutdown_connections 2024-09-13T16:50:59.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:50:59.973+0000 7fc985c00700 1 -- 172.21.15.3:0/2166871992 wait complete. 2024-09-13T16:50:59.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.6 2024-09-13T16:50:59.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-09-13T16:50:59.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.1 2024-09-13T16:50:59.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-09-13T16:50:59.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-09-13T16:50:59.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-09-13T16:51:00.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.4 2024-09-13T16:51:00.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T16:51:00.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T16:51:00.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T16:51:00.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T16:51:00.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T16:51:01.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T16:51:01.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T16:51:01.001 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T16:51:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.201+0000 7fbaa7a00700 1 Processor -- start 2024-09-13T16:51:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.201+0000 7fbaa7a00700 1 -- start start 2024-09-13T16:51:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.205+0000 7fbaa7a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbaa00ce8d0 0x7fbaa00ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.205+0000 7fbaa7a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa00cf260 con 0x7fbaa00b59a0 2024-09-13T16:51:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.205+0000 7fbaa7a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa00b2f20 con 0x7fbaa015e8e0 2024-09-13T16:51:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.205+0000 7fbaa7a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa00b5f20 con 0x7fbaa00ce8d0 2024-09-13T16:51:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.205+0000 7fbaa5a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbaa00ce8d0 0x7fbaa00ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.205+0000 7fba9fe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbaa00b59a0 0x7fbaa0161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39554/0 (socket says 172.21.15.3:39554) 2024-09-13T16:51:01.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.205+0000 7fba9fe00700 1 -- 172.21.15.3:0/2265497220 learned_addr learned my addr 172.21.15.3:0/2265497220 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:01.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300822107 0 0) 0x7fbaa00cf260 con 0x7fbaa00b59a0 2024-09-13T16:51:01.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba840035d0 con 0x7fbaa00b59a0 2024-09-13T16:51:01.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221822670 0 0) 0x7fbaa00b2f20 con 0x7fbaa015e8e0 2024-09-13T16:51:01.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaa00cf260 con 0x7fbaa015e8e0 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784647222 0 0) 0x7fba840035d0 con 0x7fbaa00b59a0 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 >> v2:172.21.15.3:3300/0 conn(0x7fbaa00ce8d0 msgr2=0x7fbaa00ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 --2- 172.21.15.3:0/2265497220 >> v2:172.21.15.3:3300/0 conn(0x7fbaa00ce8d0 0x7fbaa00ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 >> v1:172.21.15.157:6789/0 conn(0x7fbaa015e8e0 legacy=0x7fbaa0162620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa00b3200 con 0x7fbaa00b59a0 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fba8c001670 con 0x7fbaa00b59a0 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 >> v1:172.21.15.3:6789/0 conn(0x7fbaa00b59a0 legacy=0x7fbaa0161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba84004c40 0x7fba84005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84003810 0x7fba84003c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba840041d0 0x7fba8400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba900010e0 con 0x7fba84004c40 2024-09-13T16:51:01.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa00b5f20 con 0x7fba840041d0 2024-09-13T16:51:01.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.209+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8400c260 con 0x7fba84003810 2024-09-13T16:51:01.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5000700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba84004c40 0x7fba84005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fba9fe00700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84003810 0x7fba84003c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5a00700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba840041d0 0x7fba8400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5000700 1 -- 172.21.15.3:0/2265497220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84003810 msgr2=0x7fba84003c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5000700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84003810 0x7fba84003c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5000700 1 -- 172.21.15.3:0/2265497220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba840041d0 msgr2=0x7fba8400bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5000700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba840041d0 0x7fba8400bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5000700 1 -- 172.21.15.3:0/2265497220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa00b3200 con 0x7fba84004c40 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5a00700 1 -- 172.21.15.3:0/2265497220 reap_dead start 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa5000700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba84004c40 0x7fba84005020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba90007e30 tx=0x7fba840058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d99c1cf7ca0fd8f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900078d0 con 0x7fba84004c40 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba900052d0 con 0x7fba84004c40 2024-09-13T16:51:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fba9f400700 1 -- 172.21.15.3:0/2265497220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900078d0 con 0x7fba84004c40 2024-09-13T16:51:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa7a00700 1 -- 172.21.15.3:0/2265497220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba84004c40 msgr2=0x7fba84005020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa7a00700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba84004c40 0x7fba84005020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba90007e30 tx=0x7fba840058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa7a00700 1 -- 172.21.15.3:0/2265497220 shutdown_connections 2024-09-13T16:51:01.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa7a00700 1 --2- 172.21.15.3:0/2265497220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba84004c40 0x7fba84005020 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.213+0000 7fbaa7a00700 1 -- 172.21.15.3:0/2265497220 >> 172.21.15.3:0/2265497220 conn(0x7fbaa0071dc0 msgr2=0x7fbaa006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.217+0000 7fbaa7a00700 1 -- 172.21.15.3:0/2265497220 shutdown_connections 2024-09-13T16:51:01.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.217+0000 7fbaa7a00700 1 -- 172.21.15.3:0/2265497220 wait complete. 2024-09-13T16:51:01.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.217+0000 7fbaa7a00700 1 Processor -- start 2024-09-13T16:51:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa7a00700 1 -- start start 2024-09-13T16:51:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa7a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84004c40 0x7fbaa014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa7a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa006b430 0x7fbaa006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa7a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaa01fa370 0x7fbaa01fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa7a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa006bdf0 con 0x7fbaa01fa370 2024-09-13T16:51:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fba9fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa006b430 0x7fbaa006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fba9fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa006b430 0x7fbaa006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52056/0 (socket says 172.21.15.3:52056) 2024-09-13T16:51:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fba9fe00700 1 -- 172.21.15.3:0/631357736 learned_addr learned my addr 172.21.15.3:0/631357736 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa014f300 con 0x7fbaa006b430 2024-09-13T16:51:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa014f5a0 con 0x7fba84004c40 2024-09-13T16:51:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa5000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84004c40 0x7fbaa014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa5a00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaa01fa370 0x7fbaa01fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fba9fe00700 1 -- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84004c40 msgr2=0x7fbaa014edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fba9fe00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84004c40 0x7fbaa014edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fba9fe00700 1 -- 172.21.15.3:0/631357736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaa01fa370 msgr2=0x7fbaa01fa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fba9fe00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaa01fa370 0x7fbaa01fa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fba9fe00700 1 -- 172.21.15.3:0/631357736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa014f880 con 0x7fbaa006b430 2024-09-13T16:51:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.221+0000 7fbaa5a00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaa01fa370 0x7fbaa01fa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:01.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.225+0000 7fba9fe00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa006b430 0x7fbaa006b8b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0010e0 tx=0x7fba8c007670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.225+0000 7fba9cc00700 1 -- 172.21.15.3:0/631357736 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c007850 con 0x7fbaa006b430 2024-09-13T16:51:01.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.225+0000 7fba9cc00700 1 -- 172.21.15.3:0/631357736 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c00f850 con 0x7fbaa006b430 2024-09-13T16:51:01.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.225+0000 7fba9cc00700 1 -- 172.21.15.3:0/631357736 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00fc20 con 0x7fbaa006b430 2024-09-13T16:51:01.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.225+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa014fb60 con 0x7fbaa006b430 2024-09-13T16:51:01.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.225+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa01fad60 con 0x7fbaa006b430 2024-09-13T16:51:01.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.225+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa00adc70 con 0x7fbaa006b430 2024-09-13T16:51:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.229+0000 7fba9cc00700 1 -- 172.21.15.3:0/631357736 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba8c00fd80 con 0x7fbaa006b430 2024-09-13T16:51:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.229+0000 7fba9cc00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fba80066760 0x7fba80068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.229+0000 7fba9cc00700 1 -- 172.21.15.3:0/631357736 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fba8c0849f0 con 0x7fbaa006b430 2024-09-13T16:51:01.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.233+0000 7fbaa5000700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fba80066760 0x7fba80068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.233+0000 7fbaa5000700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fba80066760 0x7fba80068c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba90015040 tx=0x7fba90005000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.241+0000 7fba9cc00700 1 -- 172.21.15.3:0/631357736 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fba8c051ad0 con 0x7fbaa006b430 2024-09-13T16:51:01.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.253+0000 7f1776400700 1 Processor -- start 2024-09-13T16:51:01.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.253+0000 7f1776400700 1 -- start start 2024-09-13T16:51:01.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f1776400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1770141c20 0x7f1770070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.253+0000 7f7d39a00700 1 Processor -- start 2024-09-13T16:51:01.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.253+0000 7f7d39a00700 1 -- start start 2024-09-13T16:51:01.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d39a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d34155250 0x7f7d340b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d39a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d3406fa00 0x7f7d3406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d39a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3415c0e0 con 0x7f7d3415b070 2024-09-13T16:51:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d39a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d34155630 con 0x7f7d3406fa00 2024-09-13T16:51:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d39a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340703d0 con 0x7f7d34155250 2024-09-13T16:51:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d32a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d34155250 0x7f7d340b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d32a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d34155250 0x7f7d340b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52792/0 (socket says 172.21.15.3:52792) 2024-09-13T16:51:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d32a00700 1 -- 172.21.15.3:0/2195997933 learned_addr learned my addr 172.21.15.3:0/2195997933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f1776400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f17700b39e0 0x7f177006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7fbf4ce00700 1 Processor -- start 2024-09-13T16:51:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d33400700 1 --2- 172.21.15.3:0/2195997933 >> v2:172.21.15.157:3300/0 conn(0x7f7d3406fa00 0x7f7d3406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d32a00700 1 -- 172.21.15.3:0/2195997933 >> v2:172.21.15.157:3300/0 conn(0x7f7d3406fa00 msgr2=0x7f7d3406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7fbf4ce00700 1 -- start start 2024-09-13T16:51:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f1776400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1770142000 con 0x7f17700705b0 2024-09-13T16:51:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f1776400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f177006c930 con 0x7f1770141c20 2024-09-13T16:51:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f176ea00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f17700705b0 0x7f17700b32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37586/0 (socket says 172.21.15.3:37586) 2024-09-13T16:51:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f176ea00700 1 -- 172.21.15.3:0/2336267547 learned_addr learned my addr 172.21.15.3:0/2336267547 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f1776400700 1 -- 172.21.15.3:0/2336267547 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17700709f0 con 0x7f17700b39e0 2024-09-13T16:51:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f176f400700 1 --2- 172.21.15.3:0/2336267547 >> v2:172.21.15.3:3300/0 conn(0x7f1770141c20 0x7f1770070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f176fe00700 1 --2- 172.21.15.3:0/2336267547 >> v2:172.21.15.157:3300/0 conn(0x7f17700b39e0 0x7f177006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d32a00700 1 --2- 172.21.15.3:0/2195997933 >> v2:172.21.15.157:3300/0 conn(0x7f7d3406fa00 0x7f7d3406fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d32a00700 1 -- 172.21.15.3:0/2195997933 >> v1:172.21.15.157:6789/0 conn(0x7f7d3415b070 legacy=0x7f7d340cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d32a00700 1 -- 172.21.15.3:0/2195997933 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340b4000 con 0x7f7d34155250 2024-09-13T16:51:01.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d33400700 1 --2- 172.21.15.3:0/2195997933 >> v2:172.21.15.157:3300/0 conn(0x7f7d3406fa00 0x7f7d3406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f176e000700 1 -- 172.21.15.3:0/2336267547 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651925636 0 0) 0x7f175c004ab0 con 0x7f17700705b0 2024-09-13T16:51:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f176e000700 1 -- 172.21.15.3:0/2336267547 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17540035d0 con 0x7f17700705b0 2024-09-13T16:51:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7fdbd1c00700 1 Processor -- start 2024-09-13T16:51:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f7d32a00700 1 --2- 172.21.15.3:0/2195997933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d34155250 0x7f7d340b5c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009e50 tx=0x7f7d2c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d16e3ed1f40caa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f176f400700 1 -- 172.21.15.3:0/2336267547 >> v2:172.21.15.157:3300/0 conn(0x7f17700b39e0 msgr2=0x7f177006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7f176e000700 1 -- 172.21.15.3:0/2336267547 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4009875721 0 0) 0x7f17540035d0 con 0x7f17700705b0 2024-09-13T16:51:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf4ce00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf48141c20 0x7fbf480b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f176f400700 1 --2- 172.21.15.3:0/2336267547 >> v2:172.21.15.157:3300/0 conn(0x7f17700b39e0 0x7f177006cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f176f400700 1 -- 172.21.15.3:0/2336267547 >> v1:172.21.15.157:6789/0 conn(0x7f17700705b0 legacy=0x7f17700b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f176f400700 1 -- 172.21.15.3:0/2336267547 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1770070be0 con 0x7f1770141c20 2024-09-13T16:51:01.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf4ce00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf4806dd60 0x7fbf480b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf4ce00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf48142000 con 0x7fbf480b33c0 2024-09-13T16:51:01.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf4ce00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4806c930 con 0x7fbf48141c20 2024-09-13T16:51:01.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf4ce00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf48070be0 con 0x7fbf4806dd60 2024-09-13T16:51:01.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f176fe00700 1 --2- 172.21.15.3:0/2336267547 >> v2:172.21.15.157:3300/0 conn(0x7f17700b39e0 0x7f177006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:01.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.257+0000 7fdbd1c00700 1 -- start start 2024-09-13T16:51:01.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf45400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbf480b33c0 0x7fbf4815c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39564/0 (socket says 172.21.15.3:39564) 2024-09-13T16:51:01.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf45400700 1 -- 172.21.15.3:0/120133678 learned_addr learned my addr 172.21.15.3:0/120133678 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:01.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f176f400700 1 --2- 172.21.15.3:0/2336267547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770070070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f177015c8d0 tx=0x7f1760009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f193dd847f698f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f7d31600700 1 -- 172.21.15.3:0/2195997933 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00e070 con 0x7f7d34155250 2024-09-13T16:51:01.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f7d31600700 1 -- 172.21.15.3:0/2195997933 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c00b9c0 con 0x7f7d34155250 2024-09-13T16:51:01.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f7d31600700 1 -- 172.21.15.3:0/2195997933 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00bd70 con 0x7f7d34155250 2024-09-13T16:51:01.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf46800700 1 --2- 172.21.15.3:0/120133678 >> v2:172.21.15.157:3300/0 conn(0x7fbf4806dd60 0x7fbf480b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf45e00700 1 --2- 172.21.15.3:0/120133678 >> v2:172.21.15.3:3300/0 conn(0x7fbf48141c20 0x7fbf480b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf46800700 1 -- 172.21.15.3:0/120133678 >> v2:172.21.15.3:3300/0 conn(0x7fbf48141c20 msgr2=0x7fbf480b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf46800700 1 --2- 172.21.15.3:0/120133678 >> v2:172.21.15.3:3300/0 conn(0x7fbf48141c20 0x7fbf480b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf46800700 1 -- 172.21.15.3:0/120133678 >> v1:172.21.15.3:6789/0 conn(0x7fbf480b33c0 legacy=0x7fbf4815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf46800700 1 -- 172.21.15.3:0/120133678 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48070ec0 con 0x7fbf4806dd60 2024-09-13T16:51:01.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fdbd1c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdbcc142440 0x7fdbcc070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fdbd1c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbcc15c820 0x7fdbcc159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7ff22ec00700 1 Processor -- start 2024-09-13T16:51:01.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fdbd1c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbcc15cd90 con 0x7fdbcc0cc110 2024-09-13T16:51:01.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fdbd1c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbcc0b1d70 con 0x7fdbcc15c820 2024-09-13T16:51:01.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fdbd1c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbcc070590 con 0x7fdbcc142440 2024-09-13T16:51:01.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf46800700 1 --2- 172.21.15.3:0/120133678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480b3830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf480b1e20 tx=0x7fbf4000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900a6a33f75dae53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7ff22ec00700 1 -- start start 2024-09-13T16:51:01.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf44a00700 1 -- 172.21.15.3:0/120133678 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4000e070 con 0x7fbf4806dd60 2024-09-13T16:51:01.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf44a00700 1 -- 172.21.15.3:0/120133678 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4000bbd0 con 0x7fbf4806dd60 2024-09-13T16:51:01.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fbf44a00700 1 -- 172.21.15.3:0/120133678 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4000e070 con 0x7fbf4806dd60 2024-09-13T16:51:01.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f7d39a00700 1 -- 172.21.15.3:0/2195997933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d34155250 msgr2=0x7f7d340b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2195997933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d34155250 0x7f7d340b5c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009e50 tx=0x7f7d2c00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:01.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fdbcb400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdbcc0cc110 0x7fdbcc0b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39572/0 (socket says 172.21.15.3:39572) 2024-09-13T16:51:01.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fdbcb400700 1 -- 172.21.15.3:0/1938949470 learned_addr learned my addr 172.21.15.3:0/1938949470 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:01.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f176e000700 1 -- 172.21.15.3:0/2336267547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c004fa0 con 0x7f1770141c20 2024-09-13T16:51:01.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7f176e000700 1 -- 172.21.15.3:0/2336267547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1760004030 con 0x7f1770141c20 2024-09-13T16:51:01.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.261+0000 7fdbcbe00700 1 --2- 172.21.15.3:0/1938949470 >> v2:172.21.15.157:3300/0 conn(0x7fdbcc142440 0x7fdbcc070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f176e000700 1 -- 172.21.15.3:0/2336267547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17600043a0 con 0x7f1770141c20 2024-09-13T16:51:01.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbcaa00700 1 -- 172.21.15.3:0/1938949470 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795394619 0 0) 0x7fdbcc15cd90 con 0x7fdbcc0cc110 2024-09-13T16:51:01.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1200700 1 --2- 172.21.15.3:0/1938949470 >> v2:172.21.15.3:3300/0 conn(0x7fdbcc15c820 0x7fdbcc159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbcaa00700 1 -- 172.21.15.3:0/1938949470 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb00057e0 con 0x7fdbcc0cc110 2024-09-13T16:51:01.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f7d39a00700 1 -- 172.21.15.3:0/2195997933 shutdown_connections 2024-09-13T16:51:01.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fbf4ce00700 1 -- 172.21.15.3:0/120133678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 msgr2=0x7fbf480b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/120133678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480b3830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf480b1e20 tx=0x7fbf4000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:01.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f1776400700 1 -- 172.21.15.3:0/2336267547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 msgr2=0x7f1770070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f1776400700 1 --2- 172.21.15.3:0/2336267547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770070070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f177015c8d0 tx=0x7f1760009d90 comp rx=0 tx=0).stop 2024-09-13T16:51:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2195997933 >> v2:172.21.15.157:3300/0 conn(0x7f7d3406fa00 0x7f7d3406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2195997933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d34155250 0x7f7d340b5c50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f7d39a00700 1 -- 172.21.15.3:0/2195997933 >> 172.21.15.3:0/2195997933 conn(0x7f7d340b2ad0 msgr2=0x7f7d340b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbcaa00700 1 -- 172.21.15.3:0/1938949470 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096883745 0 0) 0x7fdbb00057e0 con 0x7fdbcc0cc110 2024-09-13T16:51:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1200700 1 -- 172.21.15.3:0/1938949470 >> v2:172.21.15.157:3300/0 conn(0x7fdbcc142440 msgr2=0x7fdbcc070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1200700 1 --2- 172.21.15.3:0/1938949470 >> v2:172.21.15.157:3300/0 conn(0x7fdbcc142440 0x7fdbcc070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1200700 1 -- 172.21.15.3:0/1938949470 >> v1:172.21.15.3:6789/0 conn(0x7fdbcc0cc110 legacy=0x7fdbcc0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1200700 1 -- 172.21.15.3:0/1938949470 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc15d070 con 0x7fdbcc15c820 2024-09-13T16:51:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22ec00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff2280b3cd0 0x7ff228071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fbf4ce00700 1 -- 172.21.15.3:0/120133678 shutdown_connections 2024-09-13T16:51:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/120133678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480b3830 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/120133678 >> v2:172.21.15.3:3300/0 conn(0x7fbf48141c20 0x7fbf480b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fbf4ce00700 1 -- 172.21.15.3:0/120133678 >> 172.21.15.3:0/120133678 conn(0x7fbf4806b1e0 msgr2=0x7fbf4806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f1776400700 1 -- 172.21.15.3:0/2336267547 shutdown_connections 2024-09-13T16:51:01.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f1776400700 1 --2- 172.21.15.3:0/2336267547 >> v2:172.21.15.157:3300/0 conn(0x7f17700b39e0 0x7f177006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f1776400700 1 --2- 172.21.15.3:0/2336267547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770070070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f1776400700 1 -- 172.21.15.3:0/2336267547 >> 172.21.15.3:0/2336267547 conn(0x7f177006b1e0 msgr2=0x7f177006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f1776400700 1 -- 172.21.15.3:0/2336267547 shutdown_connections 2024-09-13T16:51:01.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7f1776400700 1 -- 172.21.15.3:0/2336267547 wait complete. 2024-09-13T16:51:01.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22ec00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2280b40b0 con 0x7ff22806b4a0 2024-09-13T16:51:01.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22ec00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22806e170 con 0x7ff22815a540 2024-09-13T16:51:01.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22ec00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22815aa60 con 0x7ff2280b3cd0 2024-09-13T16:51:01.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1200700 1 --2- 172.21.15.3:0/1938949470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc15c820 0x7fdbcc159cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0ce540 tx=0x7fdbc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed643bbbe98e3dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbcaa00700 1 -- 172.21.15.3:0/1938949470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400e070 con 0x7fdbcc15c820 2024-09-13T16:51:01.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22ce00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff22806b4a0 0x7ff228159e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39588/0 (socket says 172.21.15.3:39588) 2024-09-13T16:51:01.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbcaa00700 1 -- 172.21.15.3:0/1938949470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc4003a40 con 0x7fdbcc15c820 2024-09-13T16:51:01.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fbf4ce00700 1 -- 172.21.15.3:0/120133678 shutdown_connections 2024-09-13T16:51:01.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fbf4ce00700 1 -- 172.21.15.3:0/120133678 wait complete. 2024-09-13T16:51:01.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbcaa00700 1 -- 172.21.15.3:0/1938949470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8001670 con 0x7fdbcc15c820 2024-09-13T16:51:01.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22e200700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff22815a540 0x7ff22806c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37594/0 (socket says 172.21.15.3:37594) 2024-09-13T16:51:01.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22e200700 1 -- 172.21.15.3:0/1721153361 learned_addr learned my addr 172.21.15.3:0/1721153361 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22d800700 1 --2- 172.21.15.3:0/1721153361 >> v2:172.21.15.3:3300/0 conn(0x7ff2280b3cd0 0x7ff228071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff227e00700 1 -- 172.21.15.3:0/1721153361 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076698029 0 0) 0x7ff22806e170 con 0x7ff22815a540 2024-09-13T16:51:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff227e00700 1 -- 172.21.15.3:0/1721153361 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2100057e0 con 0x7ff22815a540 2024-09-13T16:51:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1c00700 1 -- 172.21.15.3:0/1938949470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc15c820 msgr2=0x7fdbcc159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/1938949470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc15c820 0x7fdbcc159cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0ce540 tx=0x7fdbc400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff227e00700 1 -- 172.21.15.3:0/1721153361 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671969536 0 0) 0x7ff2280b40b0 con 0x7ff22806b4a0 2024-09-13T16:51:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff227e00700 1 -- 172.21.15.3:0/1721153361 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff22806e170 con 0x7ff22806b4a0 2024-09-13T16:51:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1c00700 1 -- 172.21.15.3:0/1938949470 shutdown_connections 2024-09-13T16:51:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/1938949470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc15c820 0x7fdbcc159cd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/1938949470 >> v2:172.21.15.157:3300/0 conn(0x7fdbcc142440 0x7fdbcc070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7fdbd1c00700 1 -- 172.21.15.3:0/1938949470 >> 172.21.15.3:0/1938949470 conn(0x7fdbcc0cb620 msgr2=0x7fdbcc0cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.265+0000 7ff22d800700 1 -- 172.21.15.3:0/1721153361 >> v1:172.21.15.157:6789/0 conn(0x7ff22815a540 legacy=0x7ff22806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f1776400700 1 Processor -- start 2024-09-13T16:51:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f1776400700 1 -- start start 2024-09-13T16:51:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f7d39a00700 1 -- 172.21.15.3:0/2195997933 shutdown_connections 2024-09-13T16:51:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf4ce00700 1 Processor -- start 2024-09-13T16:51:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf4ce00700 1 -- start start 2024-09-13T16:51:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff22d800700 1 -- 172.21.15.3:0/1721153361 >> v1:172.21.15.3:6789/0 conn(0x7ff22806b4a0 legacy=0x7ff228159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff22d800700 1 -- 172.21.15.3:0/1721153361 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22815ad40 con 0x7ff2280b3cd0 2024-09-13T16:51:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff22d800700 1 --2- 172.21.15.3:0/1721153361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2280b3cd0 0x7ff228071ff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009850 tx=0x7ff21c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c630d3744654541 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f7d39a00700 1 -- 172.21.15.3:0/2195997933 wait complete. 2024-09-13T16:51:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fdbd1c00700 1 -- 172.21.15.3:0/1938949470 shutdown_connections 2024-09-13T16:51:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff227e00700 1 -- 172.21.15.3:0/1721153361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c011830 con 0x7ff2280b3cd0 2024-09-13T16:51:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff227e00700 1 -- 172.21.15.3:0/1721153361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c011990 con 0x7ff2280b3cd0 2024-09-13T16:51:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff227e00700 1 -- 172.21.15.3:0/1721153361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c011830 con 0x7ff2280b3cd0 2024-09-13T16:51:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf4ce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf4ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf480b33c0 0x7fbf480c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf4ce00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48141c20 0x7fbf481f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff22ec00700 1 -- 172.21.15.3:0/1721153361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2280b3cd0 msgr2=0x7ff228071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff22ec00700 1 --2- 172.21.15.3:0/1721153361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2280b3cd0 0x7ff228071ff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009850 tx=0x7ff21c009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f1776400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17700705b0 0x7f177013ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f1776400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17700b39e0 0x7f17700c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f1776400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f7d39a00700 1 Processor -- start 2024-09-13T16:51:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf4ce00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480c66d0 con 0x7fbf480b33c0 2024-09-13T16:51:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf4ce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480c69b0 con 0x7fbf4806dd60 2024-09-13T16:51:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf45400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf480b33c0 0x7fbf480c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf45e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fdbd1c00700 1 -- 172.21.15.3:0/1938949470 wait complete. 2024-09-13T16:51:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f1776400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17700c5300 con 0x7f1770141c20 2024-09-13T16:51:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f1776400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17700c4a60 con 0x7f17700705b0 2024-09-13T16:51:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f1776400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17700c4d40 con 0x7f17700b39e0 2024-09-13T16:51:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54722/0 (socket says 172.21.15.3:54722) 2024-09-13T16:51:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176fe00700 1 -- 172.21.15.3:0/2782060092 learned_addr learned my addr 172.21.15.3:0/2782060092 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fe08ac00700 1 Processor -- start 2024-09-13T16:51:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176f400700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17700705b0 0x7f177013ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f7d39a00700 1 -- start start 2024-09-13T16:51:01.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf45e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52104/0 (socket says 172.21.15.3:52104) 2024-09-13T16:51:01.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf45e00700 1 -- 172.21.15.3:0/1393277066 learned_addr learned my addr 172.21.15.3:0/1393277066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480c6c90 con 0x7fbf48141c20 2024-09-13T16:51:01.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fbf46800700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48141c20 0x7fbf481f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176fe00700 1 -- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17700b39e0 msgr2=0x7f17700c61b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T16:51:01.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176fe00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17700b39e0 0x7f17700c61b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176fe00700 1 -- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17700705b0 msgr2=0x7f177013ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176fe00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17700705b0 0x7f177013ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7f176fe00700 1 -- 172.21.15.3:0/2782060092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1768009a70 con 0x7f1770141c20 2024-09-13T16:51:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff22ec00700 1 -- 172.21.15.3:0/1721153361 shutdown_connections 2024-09-13T16:51:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff22ec00700 1 --2- 172.21.15.3:0/1721153361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2280b3cd0 0x7ff228071ff0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7ff22ec00700 1 -- 172.21.15.3:0/1721153361 >> 172.21.15.3:0/1721153361 conn(0x7ff2280b0c50 msgr2=0x7ff2280b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.269+0000 7fe08ac00700 1 -- start start 2024-09-13T16:51:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf45e00700 1 -- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48141c20 msgr2=0x7fbf481f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf45e00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48141c20 0x7fbf481f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf45e00700 1 -- 172.21.15.3:0/1393277066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf480b33c0 msgr2=0x7fbf480c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf45e00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf480b33c0 0x7fbf480c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf45e00700 1 -- 172.21.15.3:0/1393277066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40009a70 con 0x7fbf4806dd60 2024-09-13T16:51:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1c00700 1 Processor -- start 2024-09-13T16:51:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1c00700 1 -- start start 2024-09-13T16:51:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7ff22ec00700 1 -- 172.21.15.3:0/1721153361 shutdown_connections 2024-09-13T16:51:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf45e00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480ca4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800d950 tx=0x7fbf3800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f176fe00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770157870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f176800efb0 tx=0x7f176800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7ff454600700 1 Processor -- start 2024-09-13T16:51:01.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7ff22ec00700 1 -- 172.21.15.3:0/1721153361 wait complete. 2024-09-13T16:51:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7ff454600700 1 -- start start 2024-09-13T16:51:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d39a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d3406fa00 0x7f7d341ed740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d39a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d34155250 0x7f7d3406c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f1767400700 1 -- 172.21.15.3:0/2782060092 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800cd40 con 0x7f1770141c20 2024-09-13T16:51:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f1767400700 1 -- 172.21.15.3:0/2782060092 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176800f040 con 0x7f1770141c20 2024-09-13T16:51:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f1767400700 1 -- 172.21.15.3:0/2782060092 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800cea0 con 0x7f1770141c20 2024-09-13T16:51:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d39a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415b070 0x7f7d3414e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d39a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3414c4f0 con 0x7f7d34155250 2024-09-13T16:51:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d39a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3406cdf0 con 0x7f7d3415b070 2024-09-13T16:51:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d39a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3414be30 con 0x7f7d3406fa00 2024-09-13T16:51:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d33400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415b070 0x7f7d3414e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d33400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415b070 0x7f7d3414e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52118/0 (socket says 172.21.15.3:52118) 2024-09-13T16:51:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d33400700 1 -- 172.21.15.3:0/2465394494 learned_addr learned my addr 172.21.15.3:0/2465394494 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf3e000700 1 -- 172.21.15.3:0/1393277066 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38006d30 con 0x7fbf4806dd60 2024-09-13T16:51:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf3e000700 1 -- 172.21.15.3:0/1393277066 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf38006e90 con 0x7fbf4806dd60 2024-09-13T16:51:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf3e000700 1 -- 172.21.15.3:0/1393277066 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800f8e0 con 0x7fbf4806dd60 2024-09-13T16:51:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf480c6f70 con 0x7fbf4806dd60 2024-09-13T16:51:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf480c7770 con 0x7fbf4806dd60 2024-09-13T16:51:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0cc110 0x7fdbcc1417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc142440 0x7fdbcc0aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f1776400700 1 -- 172.21.15.3:0/2782060092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17600094b0 con 0x7f1770141c20 2024-09-13T16:51:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f1776400700 1 -- 172.21.15.3:0/2782060092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177013b0b0 con 0x7f1770141c20 2024-09-13T16:51:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d32000700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d34155250 0x7f7d3406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15c820 0x7fdbcc0a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc13f450 con 0x7fdbcc142440 2024-09-13T16:51:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0a85d0 con 0x7fdbcc15c820 2024-09-13T16:51:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0a88b0 con 0x7fdbcc0cc110 2024-09-13T16:51:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbcbe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0cc110 0x7fdbcc1417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbcbe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0cc110 0x7fdbcc1417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52818/0 (socket says 172.21.15.3:52818) 2024-09-13T16:51:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbcbe00700 1 -- 172.21.15.3:0/3483607432 learned_addr learned my addr 172.21.15.3:0/3483607432 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d33400700 1 -- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d3406fa00 msgr2=0x7f7d341ed740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:51:01.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d33400700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d3406fa00 0x7f7d341ed740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d33400700 1 -- 172.21.15.3:0/2465394494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d34155250 msgr2=0x7f7d3406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d33400700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d34155250 0x7f7d3406c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7f7d33400700 1 -- 172.21.15.3:0/2465394494 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c009b30 con 0x7f7d3415b070 2024-09-13T16:51:01.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbcb400700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc142440 0x7fdbcc0aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fdbd1200700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15c820 0x7fdbcc0a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fe08ac00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe084070db0 0x7fe084071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fe08ac00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe08406c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fe08ac00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840b3450 con 0x7fe08415b660 2024-09-13T16:51:01.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fe08ac00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0840b40d0 con 0x7fe084070db0 2024-09-13T16:51:01.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fe08ac00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe084071780 con 0x7fe08406c540 2024-09-13T16:51:01.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f7d33400700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415b070 0x7f7d3414e510 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000e810 tx=0x7f7d2000ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.273+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf20005310 con 0x7fbf4806dd60 2024-09-13T16:51:01.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fbf3e000700 1 -- 172.21.15.3:0/1393277066 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf3800fa40 con 0x7fbf4806dd60 2024-09-13T16:51:01.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f1764c00700 1 -- 172.21.15.3:0/2782060092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1744005310 con 0x7f1770141c20 2024-09-13T16:51:01.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff454600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c141c20 0x7ff44c070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe088c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe08406c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe083e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe08415b660 0x7fe0840b2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37600/0 (socket says 172.21.15.3:37600) 2024-09-13T16:51:01.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe083e00700 1 -- 172.21.15.3:0/3499509515 learned_addr learned my addr 172.21.15.3:0/3499509515 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:01.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff454600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff44c0705b0 0x7ff44c0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbcbe00700 1 -- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15c820 msgr2=0x7fdbcc0a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbcbe00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15c820 0x7fdbcc0a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbcbe00700 1 -- 172.21.15.3:0/3483607432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc142440 msgr2=0x7fdbcc0aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbcbe00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc142440 0x7fdbcc0aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff454600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff44c0b3810 0x7ff44c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fbf3e000700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf28066780 0x7fbf28068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff454600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c06c930 con 0x7ff44c0b3810 2024-09-13T16:51:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe082a00700 1 -- 172.21.15.3:0/3499509515 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390912785 0 0) 0x7fe0840b3450 con 0x7fe08415b660 2024-09-13T16:51:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe083400700 1 --2- 172.21.15.3:0/3499509515 >> v2:172.21.15.157:3300/0 conn(0x7fe084070db0 0x7fe084071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbcbe00700 1 -- 172.21.15.3:0/3483607432 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc4009a70 con 0x7fdbcc0cc110 2024-09-13T16:51:01.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff454600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0b3cc0 con 0x7ff44c0705b0 2024-09-13T16:51:01.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fbf3e000700 1 -- 172.21.15.3:0/1393277066 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbf38055530 con 0x7fbf4806dd60 2024-09-13T16:51:01.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff454600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0709f0 con 0x7ff44c141c20 2024-09-13T16:51:01.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f7d2aa00700 1 -- 172.21.15.3:0/2465394494 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20004230 con 0x7f7d3415b070 2024-09-13T16:51:01.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff44c0705b0 0x7ff44c0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff44c0705b0 0x7ff44c0b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52152/0 (socket says 172.21.15.3:52152) 2024-09-13T16:51:01.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451200700 1 -- 172.21.15.3:0/3640116748 learned_addr learned my addr 172.21.15.3:0/3640116748 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe082a00700 1 -- 172.21.15.3:0/3499509515 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0680035d0 con 0x7fe08415b660 2024-09-13T16:51:01.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451c00700 1 --2- 172.21.15.3:0/3640116748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c141c20 0x7ff44c070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3414c110 con 0x7f7d3415b070 2024-09-13T16:51:01.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbcbe00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0cc110 0x7fdbcc1417a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00e9d0 tx=0x7fdbbc00ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fbf45400700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf28066780 0x7fbf28068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff452600700 1 --2- 172.21.15.3:0/3640116748 >> v2:172.21.15.3:3300/0 conn(0x7ff44c0b3810 0x7ff44c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f1767400700 1 -- 172.21.15.3:0/2782060092 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17680146a0 con 0x7f1770141c20 2024-09-13T16:51:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3406b660 con 0x7f7d3415b070 2024-09-13T16:51:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f7d2aa00700 1 -- 172.21.15.3:0/2465394494 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20004390 con 0x7f7d3415b070 2024-09-13T16:51:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f7d2aa00700 1 -- 172.21.15.3:0/2465394494 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200036c0 con 0x7f7d3415b070 2024-09-13T16:51:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbc3e00700 1 -- 172.21.15.3:0/3483607432 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc009720 con 0x7fdbcc0cc110 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbc3e00700 1 -- 172.21.15.3:0/3483607432 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc00c850 con 0x7fdbcc0cc110 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbc3e00700 1 -- 172.21.15.3:0/3483607432 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc00cbc0 con 0x7fdbcc0cc110 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbcc0a8b90 con 0x7fdbcc0cc110 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbcc0ab450 con 0x7fdbcc0cc110 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451c00700 1 -- 172.21.15.3:0/3640116748 >> v2:172.21.15.157:3300/0 conn(0x7ff44c0705b0 msgr2=0x7ff44c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451c00700 1 --2- 172.21.15.3:0/3640116748 >> v2:172.21.15.157:3300/0 conn(0x7ff44c0705b0 0x7ff44c0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451c00700 1 -- 172.21.15.3:0/3640116748 >> v2:172.21.15.3:3300/0 conn(0x7ff44c0b3810 msgr2=0x7ff44c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451c00700 1 --2- 172.21.15.3:0/3640116748 >> v2:172.21.15.3:3300/0 conn(0x7ff44c0b3810 0x7ff44c15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451c00700 1 -- 172.21.15.3:0/3640116748 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c142000 con 0x7ff44c141c20 2024-09-13T16:51:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff451c00700 1 --2- 172.21.15.3:0/3640116748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c141c20 0x7ff44c070070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff44c068770 tx=0x7ff440009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b625a1f7e8cebd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe088c00700 1 -- 172.21.15.3:0/3499509515 >> v2:172.21.15.157:3300/0 conn(0x7fe084070db0 msgr2=0x7fe084071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe088c00700 1 --2- 172.21.15.3:0/3499509515 >> v2:172.21.15.157:3300/0 conn(0x7fe084070db0 0x7fe084071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe088c00700 1 -- 172.21.15.3:0/3499509515 >> v1:172.21.15.157:6789/0 conn(0x7fe08415b660 legacy=0x7fe0840b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7fe088c00700 1 -- 172.21.15.3:0/3499509515 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840b3730 con 0x7fe08406c540 2024-09-13T16:51:01.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7ff22ec00700 1 Processor -- start 2024-09-13T16:51:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7fe088c00700 1 --2- 172.21.15.3:0/3499509515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe08406c9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009d90 tx=0x7fe07c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e141834b23230ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.277+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c005310 con 0x7f7d3415b070 2024-09-13T16:51:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7fbf45400700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf28066780 0x7fbf28068c40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf48140cd0 tx=0x7fbf34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff44be00700 1 -- 172.21.15.3:0/3640116748 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000c880 con 0x7ff44c141c20 2024-09-13T16:51:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22ec00700 1 -- start start 2024-09-13T16:51:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff44be00700 1 -- 172.21.15.3:0/3640116748 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44000c9e0 con 0x7ff44c141c20 2024-09-13T16:51:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff44be00700 1 -- 172.21.15.3:0/3640116748 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000cd70 con 0x7ff44c141c20 2024-09-13T16:51:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbcc070050 con 0x7fdbcc0cc110 2024-09-13T16:51:01.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7f1767400700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1750066760 0x7f1750068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7f1767400700 1 -- 172.21.15.3:0/2782060092 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1768085090 con 0x7f1770141c20 2024-09-13T16:51:01.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 -- 172.21.15.3:0/3640116748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c141c20 msgr2=0x7ff44c070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 --2- 172.21.15.3:0/3640116748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c141c20 0x7ff44c070070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff44c068770 tx=0x7ff440009540 comp rx=0 tx=0).stop 2024-09-13T16:51:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7fe082a00700 1 -- 172.21.15.3:0/3499509515 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00e070 con 0x7fe08406c540 2024-09-13T16:51:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 -- 172.21.15.3:0/3640116748 shutdown_connections 2024-09-13T16:51:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 --2- 172.21.15.3:0/3640116748 >> v2:172.21.15.3:3300/0 conn(0x7ff44c0b3810 0x7ff44c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 --2- 172.21.15.3:0/3640116748 >> v2:172.21.15.157:3300/0 conn(0x7ff44c0705b0 0x7ff44c0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 --2- 172.21.15.3:0/3640116748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c141c20 0x7ff44c070070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 -- 172.21.15.3:0/3640116748 >> 172.21.15.3:0/3640116748 conn(0x7ff44c06b1e0 msgr2=0x7ff44c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 -- 172.21.15.3:0/3640116748 shutdown_connections 2024-09-13T16:51:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7fe082a00700 1 -- 172.21.15.3:0/3499509515 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00bb20 con 0x7fe08406c540 2024-09-13T16:51:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7fe082a00700 1 -- 172.21.15.3:0/3499509515 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c012430 con 0x7fe08406c540 2024-09-13T16:51:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22806b4a0 0x7ff2281565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22ec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2280b3cd0 0x7ff2280cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22ec00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff22815a540 0x7ff2280cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22ec00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280cc500 con 0x7ff22815a540 2024-09-13T16:51:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22ec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22809c330 con 0x7ff22806b4a0 2024-09-13T16:51:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22ec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22809c580 con 0x7ff2280b3cd0 2024-09-13T16:51:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff22815a540 0x7ff2280cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff454600700 1 -- 172.21.15.3:0/3640116748 wait complete. 2024-09-13T16:51:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff22815a540 0x7ff2280cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54758/0 (socket says 172.21.15.3:54758) 2024-09-13T16:51:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22e200700 1 -- 172.21.15.3:0/3379732944 learned_addr learned my addr 172.21.15.3:0/3379732944 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22ce00700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2280b3cd0 0x7ff2280cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7ff22d800700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22806b4a0 0x7ff2281565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7fe08ac00700 1 -- 172.21.15.3:0/3499509515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 msgr2=0x7fe08406c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7fe08ac00700 1 --2- 172.21.15.3:0/3499509515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe08406c9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009d90 tx=0x7fe07c00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.281+0000 7f176f400700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1750066760 0x7f1750068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fdbc3e00700 1 -- 172.21.15.3:0/3483607432 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdbbc00cd20 con 0x7fdbcc0cc110 2024-09-13T16:51:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7f176f400700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1750066760 0x7f1750068c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1760000c00 tx=0x7f1760008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff22e200700 1 -- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2280b3cd0 msgr2=0x7ff2280cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff22e200700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2280b3cd0 0x7ff2280cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff22e200700 1 -- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22806b4a0 msgr2=0x7ff2281565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff22e200700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22806b4a0 0x7ff2281565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff22e200700 1 -- 172.21.15.3:0/3379732944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0094b0 con 0x7ff22815a540 2024-09-13T16:51:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fe08ac00700 1 -- 172.21.15.3:0/3499509515 shutdown_connections 2024-09-13T16:51:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fe08ac00700 1 --2- 172.21.15.3:0/3499509515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe08406c9f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fe08ac00700 1 --2- 172.21.15.3:0/3499509515 >> v2:172.21.15.157:3300/0 conn(0x7fe084070db0 0x7fe084071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fe08ac00700 1 -- 172.21.15.3:0/3499509515 >> 172.21.15.3:0/3499509515 conn(0x7fe08406a390 msgr2=0x7fe08406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fdbc3e00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdbac066780 0x7fdbac068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff22e200700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff22815a540 0x7ff2280cd110 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff22000cba0 tx=0x7ff22000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fe08ac00700 1 -- 172.21.15.3:0/3499509515 shutdown_connections 2024-09-13T16:51:01.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fdbc3e00700 1 -- 172.21.15.3:0/3483607432 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdbbc084ca0 con 0x7fdbcc0cc110 2024-09-13T16:51:01.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fe08ac00700 1 -- 172.21.15.3:0/3499509515 wait complete. 2024-09-13T16:51:01.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff454600700 1 Processor -- start 2024-09-13T16:51:01.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fdbcb400700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdbac066780 0x7fdbac068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff454600700 1 -- start start 2024-09-13T16:51:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7fdbcb400700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdbac066780 0x7fdbac068c40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001eb0 tx=0x7fdbb8001ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff454600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c0705b0 0x7ff44c13f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff225600700 1 -- 172.21.15.3:0/3379732944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220007480 con 0x7ff22815a540 2024-09-13T16:51:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff225600700 1 -- 172.21.15.3:0/3379732944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2200075e0 con 0x7ff22815a540 2024-09-13T16:51:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff225600700 1 -- 172.21.15.3:0/3379732944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220007950 con 0x7ff22815a540 2024-09-13T16:51:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22809c860 con 0x7ff22815a540 2024-09-13T16:51:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7f1767400700 1 -- 172.21.15.3:0/2782060092 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1768052100 con 0x7f1770141c20 2024-09-13T16:51:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff454600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c0b3810 0x7ff44c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff454600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c141c20 0x7ff44c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff454600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0c7e80 con 0x7ff44c141c20 2024-09-13T16:51:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff454600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0c6de0 con 0x7ff44c0705b0 2024-09-13T16:51:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff454600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0c70c0 con 0x7ff44c0b3810 2024-09-13T16:51:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff451c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c0705b0 0x7ff44c13f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff451c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c0705b0 0x7ff44c13f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52164/0 (socket says 172.21.15.3:52164) 2024-09-13T16:51:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff451c00700 1 -- 172.21.15.3:0/2062354600 learned_addr learned my addr 172.21.15.3:0/2062354600 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff452600700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c141c20 0x7ff44c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.285+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2281560c0 con 0x7ff22815a540 2024-09-13T16:51:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fbf3e000700 1 -- 172.21.15.3:0/1393277066 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf38051900 con 0x7fbf4806dd60 2024-09-13T16:51:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff451200700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c0b3810 0x7ff44c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2280677c0 con 0x7ff22815a540 2024-09-13T16:51:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff451c00700 1 -- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c0b3810 msgr2=0x7ff44c0a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff451c00700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c0b3810 0x7ff44c0a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff451c00700 1 -- 172.21.15.3:0/2062354600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c141c20 msgr2=0x7ff44c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff451c00700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c141c20 0x7ff44c15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fe08ac00700 1 Processor -- start 2024-09-13T16:51:01.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff451c00700 1 -- 172.21.15.3:0/2062354600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff440009d40 con 0x7ff44c0705b0 2024-09-13T16:51:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fe08ac00700 1 -- start start 2024-09-13T16:51:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fe08ac00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe0841534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fe08ac00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084070db0 0x7fe0841f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff451c00700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c0705b0 0x7ff44c13f9d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff440009890 tx=0x7ff440004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fe083e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe0841534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fe083e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe0841534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52886/0 (socket says 172.21.15.3:52886) 2024-09-13T16:51:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fe083e00700 1 -- 172.21.15.3:0/632435923 learned_addr learned my addr 172.21.15.3:0/632435923 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7ff225600700 1 -- 172.21.15.3:0/3379732944 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff220012400 con 0x7ff22815a540 2024-09-13T16:51:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.289+0000 7fe08ac00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe08415b660 0x7fe08414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084162e20 con 0x7fe084070db0 2024-09-13T16:51:01.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084153a10 con 0x7fe08415b660 2024-09-13T16:51:01.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0841f7610 con 0x7fe08406c540 2024-09-13T16:51:01.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7f7d2aa00700 1 -- 172.21.15.3:0/2465394494 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d20003820 con 0x7f7d3415b070 2024-09-13T16:51:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7ff449600700 1 -- 172.21.15.3:0/2062354600 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44001c040 con 0x7ff44c0705b0 2024-09-13T16:51:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7ff449600700 1 -- 172.21.15.3:0/2062354600 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44000c050 con 0x7ff44c0705b0 2024-09-13T16:51:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7ff449600700 1 -- 172.21.15.3:0/2062354600 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440003b20 con 0x7ff44c0705b0 2024-09-13T16:51:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe088c00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe08415b660 0x7fe08414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe083e00700 1 -- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe08415b660 msgr2=0x7fe08414b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe083e00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe08415b660 0x7fe08414b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe083e00700 1 -- 172.21.15.3:0/632435923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084070db0 msgr2=0x7fe0841f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe083e00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084070db0 0x7fe0841f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe083e00700 1 -- 172.21.15.3:0/632435923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c009a70 con 0x7fe08406c540 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe083e00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe0841534d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c4d0 tx=0x7fe06c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe07be00700 1 -- 172.21.15.3:0/632435923 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c005d90 con 0x7fe08406c540 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe07be00700 1 -- 172.21.15.3:0/632435923 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c0070d0 con 0x7fe08406c540 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe07be00700 1 -- 172.21.15.3:0/632435923 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c007460 con 0x7fe08406c540 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7ff225600700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff208066780 0x7ff208068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7ff225600700 1 -- 172.21.15.3:0/3379732944 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff220083bf0 con 0x7ff22815a540 2024-09-13T16:51:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c0c73a0 con 0x7ff44c0705b0 2024-09-13T16:51:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c0c7960 con 0x7ff44c0705b0 2024-09-13T16:51:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe084162810 con 0x7fe08406c540 2024-09-13T16:51:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.293+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe084152770 con 0x7fe08406c540 2024-09-13T16:51:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05c005310 con 0x7fe08406c540 2024-09-13T16:51:01.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7f7d2aa00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d14066780 0x7f7d14068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7f7d2aa00700 1 -- 172.21.15.3:0/2465394494 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7d200842e0 con 0x7f7d3415b070 2024-09-13T16:51:01.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7f7d2aa00700 1 -- 172.21.15.3:0/2465394494 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7d200847d0 con 0x7f7d3415b070 2024-09-13T16:51:01.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7ff22d800700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff208066780 0x7ff208068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c071ff0 con 0x7ff44c0705b0 2024-09-13T16:51:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7f7d32a00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d14066780 0x7f7d14068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7fdbc3e00700 1 -- 172.21.15.3:0/3483607432 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdbbc084e60 con 0x7fdbcc0cc110 2024-09-13T16:51:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7ff22d800700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff208066780 0x7ff208068c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0090b0 tx=0x7ff21c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7fe07be00700 1 -- 172.21.15.3:0/632435923 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe06c0075c0 con 0x7fe08406c540 2024-09-13T16:51:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7f7d32a00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d14066780 0x7f7d14068c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00b370 tx=0x7f7d2c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.297+0000 7ff449600700 1 -- 172.21.15.3:0/2062354600 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff440003c80 con 0x7ff44c0705b0 2024-09-13T16:51:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7fe07be00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe060066780 0x7fe060068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7fe07be00700 1 -- 172.21.15.3:0/632435923 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe06c083ef0 con 0x7fe08406c540 2024-09-13T16:51:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7fe083400700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe060066780 0x7fe060068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7ff225600700 1 -- 172.21.15.3:0/3379732944 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff220050d90 con 0x7ff22815a540 2024-09-13T16:51:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7ff449600700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff42c066780 0x7ff42c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7ff449600700 1 -- 172.21.15.3:0/2062354600 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff4400866e0 con 0x7ff44c0705b0 2024-09-13T16:51:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7ff451200700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff42c066780 0x7ff42c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7fe083400700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe060066780 0x7fe060068c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe074006b40 tx=0x7fe074006ad0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.301+0000 7ff451200700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff42c066780 0x7ff42c068c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff44c1f4630 tx=0x7ff4380092e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.305+0000 7ff449600700 1 -- 172.21.15.3:0/2062354600 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff4400537d0 con 0x7ff44c0705b0 2024-09-13T16:51:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.309+0000 7fe07be00700 1 -- 172.21.15.3:0/632435923 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe06c050f50 con 0x7fe08406c540 2024-09-13T16:51:01.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.593+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbaa0073920 con 0x7fbaa006b430 2024-09-13T16:51:01.607 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2024-09-13T16:51:01.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.601+0000 7fba9cc00700 1 -- 172.21.15.3:0/631357736 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba8c0550f0 con 0x7fbaa006b430 2024-09-13T16:51:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fba80066760 msgr2=0x7fba80068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fba80066760 0x7fba80068c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba90015040 tx=0x7fba90005000 comp rx=0 tx=0).stop 2024-09-13T16:51:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa006b430 msgr2=0x7fbaa006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa006b430 0x7fbaa006b8b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0010e0 tx=0x7fba8c007670 comp rx=0 tx=0).stop 2024-09-13T16:51:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 shutdown_connections 2024-09-13T16:51:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fba80066760 0x7fba80068c20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaa01fa370 0x7fbaa01fa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa006b430 0x7fbaa006b8b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 --2- 172.21.15.3:0/631357736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba84004c40 0x7fbaa014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 >> 172.21.15.3:0/631357736 conn(0x7fbaa0071dc0 msgr2=0x7fbaa00cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 shutdown_connections 2024-09-13T16:51:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.605+0000 7fbaa7a00700 1 -- 172.21.15.3:0/631357736 wait complete. 2024-09-13T16:51:01.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.7 2024-09-13T16:51:01.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.633+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff2281562e0 con 0x7ff22815a540 2024-09-13T16:51:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff225600700 1 -- 172.21.15.3:0/3379732944 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff220017090 con 0x7ff22815a540 2024-09-13T16:51:01.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbf20005890 con 0x7fbf4806dd60 2024-09-13T16:51:01.643 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:51:01.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff208066780 msgr2=0x7ff208068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff208066780 0x7ff208068c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0090b0 tx=0x7ff21c008030 comp rx=0 tx=0).stop 2024-09-13T16:51:01.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff22815a540 msgr2=0x7ff2280cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff22815a540 0x7ff2280cd110 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff22000cba0 tx=0x7ff22000cbd0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7fbf3e000700 1 -- 172.21.15.3:0/1393277066 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf3801f030 con 0x7fbf4806dd60 2024-09-13T16:51:01.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 shutdown_connections 2024-09-13T16:51:01.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff208066780 0x7ff208068c40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff22815a540 0x7ff2280cd110 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2280b3cd0 0x7ff2280cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 --2- 172.21.15.3:0/3379732944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22806b4a0 0x7ff2281565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.637+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 >> 172.21.15.3:0/3379732944 conn(0x7ff2280b0c50 msgr2=0x7ff2280b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.644 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2024-09-13T16:51:01.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 shutdown_connections 2024-09-13T16:51:01.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7ff22ec00700 1 -- 172.21.15.3:0/3379732944 wait complete. 2024-09-13T16:51:01.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf28066780 msgr2=0x7fbf28068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf28066780 0x7fbf28068c40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf48140cd0 tx=0x7fbf34009040 comp rx=0 tx=0).stop 2024-09-13T16:51:01.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 msgr2=0x7fbf480ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480ca4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800d950 tx=0x7fbf3800dc60 comp rx=0 tx=0).stop 2024-09-13T16:51:01.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 shutdown_connections 2024-09-13T16:51:01.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf28066780 0x7fbf28068c40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48141c20 0x7fbf481f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf480b33c0 0x7fbf480c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 --2- 172.21.15.3:0/1393277066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4806dd60 0x7fbf480ca4c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 >> 172.21.15.3:0/1393277066 conn(0x7fbf4806b1e0 msgr2=0x7fbf480cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.641+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 shutdown_connections 2024-09-13T16:51:01.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.645+0000 7fbf4ce00700 1 -- 172.21.15.3:0/1393277066 wait complete. 2024-09-13T16:51:01.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.645+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdbcc0a89f0 con 0x7fdbcc0cc110 2024-09-13T16:51:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.645+0000 7fdbc3e00700 1 -- 172.21.15.3:0/3483607432 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdbcc0a89f0 con 0x7fdbcc0cc110 2024-09-13T16:51:01.651 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:51:01.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7f1764c00700 1 -- 172.21.15.3:0/2782060092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f17440055c0 con 0x7f1770141c20 2024-09-13T16:51:01.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7f1767400700 1 -- 172.21.15.3:0/2782060092 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1768018090 con 0x7f1770141c20 2024-09-13T16:51:01.654 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:51:01.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdbac066780 msgr2=0x7fdbac068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdbac066780 0x7fdbac068c40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001eb0 tx=0x7fdbb8001ce0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0cc110 msgr2=0x7fdbcc1417a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0cc110 0x7fdbcc1417a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00e9d0 tx=0x7fdbbc00ece0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 shutdown_connections 2024-09-13T16:51:01.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdbac066780 0x7fdbac068c40 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001eb0 tx=0x7fdbb8001ce0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15c820 0x7fdbcc0a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbcc142440 0x7fdbcc0aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 --2- 172.21.15.3:0/3483607432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0cc110 0x7fdbcc1417a0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00e9d0 tx=0x7fdbbc00ece0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.649+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 >> 172.21.15.3:0/3483607432 conn(0x7fdbcc0cb620 msgr2=0x7fdbcc15bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 shutdown_connections 2024-09-13T16:51:01.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7fdbd1c00700 1 -- 172.21.15.3:0/3483607432 wait complete. 2024-09-13T16:51:01.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff44c142000 con 0x7ff44c0705b0 2024-09-13T16:51:01.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 -- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1750066760 msgr2=0x7f1750068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1750066760 0x7f1750068c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1760000c00 tx=0x7f1760008030 comp rx=0 tx=0).stop 2024-09-13T16:51:01.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 -- 172.21.15.3:0/2782060092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 msgr2=0x7f1770157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770157870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f176800efb0 tx=0x7f176800c3a0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7ff449600700 1 -- 172.21.15.3:0/2062354600 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff44c142000 con 0x7ff44c0705b0 2024-09-13T16:51:01.662 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:51:01.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 -- 172.21.15.3:0/2782060092 shutdown_connections 2024-09-13T16:51:01.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1750066760 0x7f1750068c20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1770141c20 0x7f1770157870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17700b39e0 0x7f17700c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 --2- 172.21.15.3:0/2782060092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17700705b0 0x7f177013ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.653+0000 7f1764c00700 1 -- 172.21.15.3:0/2782060092 >> 172.21.15.3:0/2782060092 conn(0x7f177006b1e0 msgr2=0x7f177006cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.657+0000 7f1764c00700 1 -- 172.21.15.3:0/2782060092 shutdown_connections 2024-09-13T16:51:01.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.657+0000 7f1764c00700 1 -- 172.21.15.3:0/2782060092 wait complete. 2024-09-13T16:51:01.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.657+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7d0c005890 con 0x7f7d3415b070 2024-09-13T16:51:01.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.657+0000 7f7d2aa00700 1 -- 172.21.15.3:0/2465394494 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d200514d0 con 0x7f7d3415b070 2024-09-13T16:51:01.666 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:51:01.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.657+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff42c066780 msgr2=0x7ff42c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.657+0000 7ff454600700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff42c066780 0x7ff42c068c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff44c1f4630 tx=0x7ff4380092e0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.657+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c0705b0 msgr2=0x7ff44c13f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.657+0000 7ff454600700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c0705b0 0x7ff44c13f9d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff440009890 tx=0x7ff440004770 comp rx=0 tx=0).stop 2024-09-13T16:51:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 shutdown_connections 2024-09-13T16:51:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7ff454600700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff42c066780 0x7ff42c068c40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7ff454600700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c141c20 0x7ff44c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7ff454600700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff44c0b3810 0x7ff44c0a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7ff454600700 1 --2- 172.21.15.3:0/2062354600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c0705b0 0x7ff44c13f9d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 >> 172.21.15.3:0/2062354600 conn(0x7ff44c06b1e0 msgr2=0x7ff44c0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d14066780 msgr2=0x7f7d14068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d14066780 0x7f7d14068c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00b370 tx=0x7f7d2c009000 comp rx=0 tx=0).stop 2024-09-13T16:51:01.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415b070 msgr2=0x7f7d3414e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415b070 0x7f7d3414e510 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000e810 tx=0x7f7d2000ebd0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 shutdown_connections 2024-09-13T16:51:01.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d14066780 0x7f7d14068c40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415b070 0x7f7d3414e510 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d34155250 0x7f7d3406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 --2- 172.21.15.3:0/2465394494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d3406fa00 0x7f7d341ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 >> 172.21.15.3:0/2465394494 conn(0x7f7d340b2ad0 msgr2=0x7f7d34159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 shutdown_connections 2024-09-13T16:51:01.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 shutdown_connections 2024-09-13T16:51:01.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7ff454600700 1 -- 172.21.15.3:0/2062354600 wait complete. 2024-09-13T16:51:01.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.661+0000 7f7d39a00700 1 -- 172.21.15.3:0/2465394494 wait complete. 2024-09-13T16:51:01.674 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-13T16:51:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.665+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe05c005890 con 0x7fe08406c540 2024-09-13T16:51:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.665+0000 7fe07be00700 1 -- 172.21.15.3:0/632435923 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe06c017020 con 0x7fe08406c540 2024-09-13T16:51:01.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.4 2024-09-13T16:51:01.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe060066780 msgr2=0x7fe060068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe060066780 0x7fe060068c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe074006b40 tx=0x7fe074006ad0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 msgr2=0x7fe0841534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe0841534d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c4d0 tx=0x7fe06c00c890 comp rx=0 tx=0).stop 2024-09-13T16:51:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 shutdown_connections 2024-09-13T16:51:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe060066780 0x7fe060068c40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe08415b660 0x7fe08414b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084070db0 0x7fe0841f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 --2- 172.21.15.3:0/632435923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08406c540 0x7fe0841534d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 >> 172.21.15.3:0/632435923 conn(0x7fe08406a390 msgr2=0x7fe0840cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.669+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 shutdown_connections 2024-09-13T16:51:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:01.673+0000 7fe08ac00700 1 -- 172.21.15.3:0/632435923 wait complete. 2024-09-13T16:51:01.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.1 2024-09-13T16:51:01.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-09-13T16:51:01.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.6 2024-09-13T16:51:01.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-09-13T16:51:01.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-09-13T16:51:01.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-09-13T16:51:02.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T16:51:02.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T16:51:02.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T16:51:02.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T16:51:02.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T16:51:02.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T16:51:02.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T16:51:02.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T16:51:02.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.865+0000 7f8603c00700 1 Processor -- start 2024-09-13T16:51:02.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.865+0000 7f8603c00700 1 -- start start 2024-09-13T16:51:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.869+0000 7f8603c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85fc070260 0x7f85fc15b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.869+0000 7f85fbe00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f85fc06fdf0 0x7f85fc0b5600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37604/0 (socket says 172.21.15.3:37604) 2024-09-13T16:51:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.869+0000 7f85fbe00700 1 -- 172.21.15.3:0/446762692 learned_addr learned my addr 172.21.15.3:0/446762692 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:02.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.869+0000 7f8602600700 1 --2- 172.21.15.3:0/446762692 >> v2:172.21.15.157:3300/0 conn(0x7f85fc070260 0x7f85fc15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.869+0000 7f8603c00700 1 -- 172.21.15.3:0/446762692 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc15ade0 con 0x7f85fc158410 2024-09-13T16:51:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.869+0000 7f8603c00700 1 -- 172.21.15.3:0/446762692 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc0b4200 con 0x7f85fc06fdf0 2024-09-13T16:51:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.869+0000 7f8603c00700 1 -- 172.21.15.3:0/446762692 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc0b44e0 con 0x7f85fc070260 2024-09-13T16:51:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.869+0000 7f85fb400700 1 -- 172.21.15.3:0/446762692 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484151410 0 0) 0x7f85fc0b4200 con 0x7f85fc06fdf0 2024-09-13T16:51:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.873+0000 7f85fb400700 1 -- 172.21.15.3:0/446762692 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e40035d0 con 0x7f85fc06fdf0 2024-09-13T16:51:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.873+0000 7f8602600700 1 -- 172.21.15.3:0/446762692 >> v1:172.21.15.157:6789/0 conn(0x7f85fc06fdf0 legacy=0x7f85fc0b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.873+0000 7f8602600700 1 -- 172.21.15.3:0/446762692 >> v1:172.21.15.3:6789/0 conn(0x7f85fc158410 legacy=0x7f85fc0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.873+0000 7f8602600700 1 -- 172.21.15.3:0/446762692 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0b47c0 con 0x7f85fc070260 2024-09-13T16:51:02.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.877+0000 7f8602600700 1 --2- 172.21.15.3:0/446762692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc070260 0x7f85fc15b240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d90 tx=0x7f85f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62876befc76582e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.877+0000 7f85fb400700 1 -- 172.21.15.3:0/446762692 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400e070 con 0x7f85fc070260 2024-09-13T16:51:02.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.877+0000 7f85fb400700 1 -- 172.21.15.3:0/446762692 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f400bb20 con 0x7f85fc070260 2024-09-13T16:51:02.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.877+0000 7f85fb400700 1 -- 172.21.15.3:0/446762692 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4012430 con 0x7f85fc070260 2024-09-13T16:51:02.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.881+0000 7f8603c00700 1 -- 172.21.15.3:0/446762692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc070260 msgr2=0x7f85fc15b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.881+0000 7f8603c00700 1 --2- 172.21.15.3:0/446762692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc070260 0x7f85fc15b240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d90 tx=0x7f85f400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:02.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.881+0000 7f8603c00700 1 -- 172.21.15.3:0/446762692 shutdown_connections 2024-09-13T16:51:02.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.881+0000 7f8603c00700 1 --2- 172.21.15.3:0/446762692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc070260 0x7f85fc15b240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.881+0000 7f8603c00700 1 -- 172.21.15.3:0/446762692 >> 172.21.15.3:0/446762692 conn(0x7f85fc0b2a90 msgr2=0x7f85fc0b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:02.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.881+0000 7f8603c00700 1 -- 172.21.15.3:0/446762692 shutdown_connections 2024-09-13T16:51:02.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.881+0000 7f8603c00700 1 -- 172.21.15.3:0/446762692 wait complete. 2024-09-13T16:51:02.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8603c00700 1 Processor -- start 2024-09-13T16:51:02.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8603c00700 1 -- start start 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8603c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85fc06fdf0 0x7f85fc203230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8603c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc070260 0x7f85fc154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8603c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc158410 0x7f85fc1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8603c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc203770 con 0x7f85fc070260 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8603c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0b2650 con 0x7f85fc158410 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8603c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc156bc0 con 0x7f85fc06fdf0 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8602600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc158410 0x7f85fc1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8601200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85fc06fdf0 0x7f85fc203230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8602600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc158410 0x7f85fc1547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52192/0 (socket says 172.21.15.3:52192) 2024-09-13T16:51:02.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.885+0000 7f8602600700 1 -- 172.21.15.3:0/2743039224 learned_addr learned my addr 172.21.15.3:0/2743039224 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f85fbe00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc070260 0x7f85fc154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f8602600700 1 -- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85fc06fdf0 msgr2=0x7f85fc203230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f8602600700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85fc06fdf0 0x7f85fc203230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f8602600700 1 -- 172.21.15.3:0/2743039224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc070260 msgr2=0x7f85fc154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f8602600700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc070260 0x7f85fc154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f8602600700 1 -- 172.21.15.3:0/2743039224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4009a70 con 0x7f85fc158410 2024-09-13T16:51:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f85fbe00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc070260 0x7f85fc154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f8602600700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc158410 0x7f85fc1547b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85f4014040 tx=0x7f85f4004b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f85f8c00700 1 -- 172.21.15.3:0/2743039224 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4012430 con 0x7f85fc158410 2024-09-13T16:51:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f85f8c00700 1 -- 172.21.15.3:0/2743039224 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4020070 con 0x7f85fc158410 2024-09-13T16:51:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f85f8c00700 1 -- 172.21.15.3:0/2743039224 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4004280 con 0x7f85fc158410 2024-09-13T16:51:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc156ea0 con 0x7f85fc158410 2024-09-13T16:51:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.889+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc157460 con 0x7f85fc158410 2024-09-13T16:51:02.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.893+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc0cf0d0 con 0x7f85fc158410 2024-09-13T16:51:02.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.893+0000 7f85f8c00700 1 -- 172.21.15.3:0/2743039224 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85f40043e0 con 0x7f85fc158410 2024-09-13T16:51:02.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.893+0000 7f85f8c00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f85dc066730 0x7f85dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.897+0000 7f85f8c00700 1 -- 172.21.15.3:0/2743039224 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f85f4022070 con 0x7f85fc158410 2024-09-13T16:51:02.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.897+0000 7f8601200700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f85dc066730 0x7f85dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.897+0000 7f8601200700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f85dc066730 0x7f85dc068bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f85f0000f60 tx=0x7f85f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.905+0000 7f85f8c00700 1 -- 172.21.15.3:0/2743039224 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f85f40530f0 con 0x7f85fc158410 2024-09-13T16:51:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.909+0000 7f2821800700 1 Processor -- start 2024-09-13T16:51:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.909+0000 7f2821800700 1 -- start start 2024-09-13T16:51:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2821800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f281c070780 0x7f281c0b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2821800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c06d620 con 0x7f281c072290 2024-09-13T16:51:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f281aa00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f281c072290 0x7f281c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39604/0 (socket says 172.21.15.3:39604) 2024-09-13T16:51:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f281aa00700 1 -- 172.21.15.3:0/2073365203 learned_addr learned my addr 172.21.15.3:0/2073365203 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2821800700 1 -- 172.21.15.3:0/2073365203 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c0b3820 con 0x7f281c0b2e80 2024-09-13T16:51:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2821800700 1 -- 172.21.15.3:0/2073365203 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281c0ae520 con 0x7f281c070780 2024-09-13T16:51:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f281b400700 1 --2- 172.21.15.3:0/2073365203 >> v2:172.21.15.3:3300/0 conn(0x7f281c070780 0x7f281c0b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428918179 0 0) 0x7f281c06d620 con 0x7f281c072290 2024-09-13T16:51:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28000035d0 con 0x7f281c072290 2024-09-13T16:51:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f8993000700 1 Processor -- start 2024-09-13T16:51:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785164509 0 0) 0x7f281c0b3820 con 0x7f281c0b2e80 2024-09-13T16:51:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f281c06d620 con 0x7f281c0b2e80 2024-09-13T16:51:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746725856 0 0) 0x7f28000035d0 con 0x7f281c072290 2024-09-13T16:51:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 >> v2:172.21.15.3:3300/0 conn(0x7f281c070780 msgr2=0x7f281c0b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 --2- 172.21.15.3:0/2073365203 >> v2:172.21.15.3:3300/0 conn(0x7f281c070780 0x7f281c0b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 >> v1:172.21.15.157:6789/0 conn(0x7f281c0b2e80 legacy=0x7f281c070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c15e2a0 con 0x7f281c072290 2024-09-13T16:51:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f280c001690 con 0x7f281c072290 2024-09-13T16:51:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 >> v1:172.21.15.3:6789/0 conn(0x7f281c072290 legacy=0x7f281c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f2800003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2800004870 0x7f2800004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2800005230 0x7f2800014240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0ae520 con 0x7f2800003bb0 2024-09-13T16:51:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000148c0 con 0x7f2800004870 2024-09-13T16:51:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2800014a80 con 0x7f2800005230 2024-09-13T16:51:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.913+0000 7f8993000700 1 -- start start 2024-09-13T16:51:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281aa00700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f2800003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281a000700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2800004870 0x7f2800004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281b400700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2800005230 0x7f2800014240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281aa00700 1 -- 172.21.15.3:0/2073365203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2800005230 msgr2=0x7f2800014240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281aa00700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2800005230 0x7f2800014240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281aa00700 1 -- 172.21.15.3:0/2073365203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2800004870 msgr2=0x7f2800004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281aa00700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2800004870 0x7f2800004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281aa00700 1 -- 172.21.15.3:0/2073365203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c15e2a0 con 0x7f2800003bb0 2024-09-13T16:51:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281b400700 1 -- 172.21.15.3:0/2073365203 reap_dead start 2024-09-13T16:51:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f281aa00700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f2800003fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c340 tx=0x7f28000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8174e6993dcd438f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7fd871a00700 1 Processor -- start 2024-09-13T16:51:02.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f8993000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f898c141c20 0x7f898c070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7fd871a00700 1 -- start start 2024-09-13T16:51:02.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c00ce00 con 0x7f2800003bb0 2024-09-13T16:51:02.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f898b400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f898c0705b0 0x7f898c0b32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37622/0 (socket says 172.21.15.3:37622) 2024-09-13T16:51:02.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c005980 con 0x7f2800003bb0 2024-09-13T16:51:02.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f8991000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f898c0b39e0 0x7f898c06cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39606/0 (socket says 172.21.15.3:39606) 2024-09-13T16:51:02.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f898be00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f898c141c20 0x7f898c070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f898be00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f898c141c20 0x7f898c070070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52210/0 (socket says 172.21.15.3:52210) 2024-09-13T16:51:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f898b400700 1 -- 172.21.15.3:0/173412848 learned_addr learned my addr 172.21.15.3:0/173412848 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f2819600700 1 -- 172.21.15.3:0/2073365203 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c005d30 con 0x7f2800003bb0 2024-09-13T16:51:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.917+0000 7f8993000700 1 -- 172.21.15.3:0/173412848 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c142000 con 0x7f898c0b39e0 2024-09-13T16:51:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 -- 172.21.15.3:0/2073365203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 msgr2=0x7f2800003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f2800003fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c340 tx=0x7f28000058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fc0d4200700 1 Processor -- start 2024-09-13T16:51:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fc0d4200700 1 -- start start 2024-09-13T16:51:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fc0d4200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc0b5f70 0x7fc0cc15b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 -- 172.21.15.3:0/2073365203 shutdown_connections 2024-09-13T16:51:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 --2- 172.21.15.3:0/2073365203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f2800003fb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 -- 172.21.15.3:0/2073365203 >> 172.21.15.3:0/2073365203 conn(0x7f281c06b1e0 msgr2=0x7f281c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 -- 172.21.15.3:0/2073365203 shutdown_connections 2024-09-13T16:51:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 -- 172.21.15.3:0/2073365203 wait complete. 2024-09-13T16:51:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898aa00700 1 -- 172.21.15.3:0/173412848 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541999488 0 0) 0x7f897c001110 con 0x7f898c0b39e0 2024-09-13T16:51:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f8993000700 1 -- 172.21.15.3:0/173412848 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c070be0 con 0x7f898c0705b0 2024-09-13T16:51:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f8993000700 1 -- 172.21.15.3:0/173412848 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c06c930 con 0x7f898c141c20 2024-09-13T16:51:02.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898aa00700 1 -- 172.21.15.3:0/173412848 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89700035d0 con 0x7f898c0b39e0 2024-09-13T16:51:02.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898be00700 1 -- 172.21.15.3:0/173412848 >> v1:172.21.15.157:6789/0 conn(0x7f898c0705b0 legacy=0x7f898c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898be00700 1 -- 172.21.15.3:0/173412848 >> v1:172.21.15.3:6789/0 conn(0x7f898c0b39e0 legacy=0x7f898c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898be00700 1 -- 172.21.15.3:0/173412848 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c070ec0 con 0x7f898c141c20 2024-09-13T16:51:02.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898be00700 1 --2- 172.21.15.3:0/173412848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 0x7f898c070070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89780097d0 tx=0x7f8978009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05aa2a93b73381e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c0cca20 0x7fd86c070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd86c06bab0 0x7fd86c070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86c06c3a0 con 0x7fd86c15a7c0 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86c071110 con 0x7fd86c06bab0 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c06c5f0 con 0x7fd86c0cca20 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd86be00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd86c15a7c0 0x7fd86c06b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37636/0 (socket says 172.21.15.3:37636) 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd86be00700 1 -- 172.21.15.3:0/3096887657 learned_addr learned my addr 172.21.15.3:0/3096887657 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd86b400700 1 --2- 172.21.15.3:0/3096887657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c0cca20 0x7fd86c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871000700 1 --2- 172.21.15.3:0/3096887657 >> v2:172.21.15.157:3300/0 conn(0x7fd86c06bab0 0x7fd86c070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd86aa00700 1 -- 172.21.15.3:0/3096887657 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201600836 0 0) 0x7fd86c06c3a0 con 0x7fd86c15a7c0 2024-09-13T16:51:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd86aa00700 1 -- 172.21.15.3:0/3096887657 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8500035d0 con 0x7fd86c15a7c0 2024-09-13T16:51:02.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871000700 1 -- 172.21.15.3:0/3096887657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c0cca20 msgr2=0x7fd86c070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:02.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871000700 1 --2- 172.21.15.3:0/3096887657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c0cca20 0x7fd86c070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871000700 1 -- 172.21.15.3:0/3096887657 >> v1:172.21.15.157:6789/0 conn(0x7fd86c15a7c0 legacy=0x7fd86c06b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871000700 1 -- 172.21.15.3:0/3096887657 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c06c8d0 con 0x7fd86c06bab0 2024-09-13T16:51:02.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd871000700 1 --2- 172.21.15.3:0/3096887657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c06bab0 0x7fd86c070bd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd864009d90 tx=0x7fd86400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f45b0d853a49bdf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd86aa00700 1 -- 172.21.15.3:0/3096887657 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400e070 con 0x7fd86c06bab0 2024-09-13T16:51:02.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd86aa00700 1 -- 172.21.15.3:0/3096887657 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86400bb20 con 0x7fd86c06bab0 2024-09-13T16:51:02.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fd86aa00700 1 -- 172.21.15.3:0/3096887657 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864012430 con 0x7fd86c06bab0 2024-09-13T16:51:02.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 Processor -- start 2024-09-13T16:51:02.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fc0d1800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc0cc06fb50 0x7fc0cc0b4690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39616/0 (socket says 172.21.15.3:39616) 2024-09-13T16:51:02.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7fc0d1800700 1 -- 172.21.15.3:0/1441923877 learned_addr learned my addr 172.21.15.3:0/1441923877 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f2821800700 1 -- start start 2024-09-13T16:51:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d0e00700 1 --2- 172.21.15.3:0/1441923877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc0b5f70 0x7fc0cc15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898aa00700 1 -- 172.21.15.3:0/173412848 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978004230 con 0x7f898c141c20 2024-09-13T16:51:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898aa00700 1 -- 172.21.15.3:0/173412848 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978004390 con 0x7f898c141c20 2024-09-13T16:51:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.921+0000 7f898aa00700 1 -- 172.21.15.3:0/173412848 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89780036c0 con 0x7f898c141c20 2024-09-13T16:51:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f8993000700 1 -- 172.21.15.3:0/173412848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 msgr2=0x7f898c070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f8993000700 1 --2- 172.21.15.3:0/173412848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 0x7f898c070070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89780097d0 tx=0x7f8978009ae0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d4200700 1 --2- 172.21.15.3:0/1441923877 >> v2:172.21.15.157:3300/0 conn(0x7fc0cc15b540 0x7fc0cc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f8993000700 1 -- 172.21.15.3:0/173412848 shutdown_connections 2024-09-13T16:51:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f8993000700 1 --2- 172.21.15.3:0/173412848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 0x7f898c070070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f8993000700 1 -- 172.21.15.3:0/173412848 >> 172.21.15.3:0/173412848 conn(0x7f898c06b1e0 msgr2=0x7f898c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f8993000700 1 -- 172.21.15.3:0/173412848 shutdown_connections 2024-09-13T16:51:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fd871a00700 1 -- 172.21.15.3:0/3096887657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c06bab0 msgr2=0x7fd86c070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fd871a00700 1 --2- 172.21.15.3:0/3096887657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c06bab0 0x7fd86c070bd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd864009d90 tx=0x7fd86400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f8993000700 1 -- 172.21.15.3:0/173412848 wait complete. 2024-09-13T16:51:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d4200700 1 -- 172.21.15.3:0/1441923877 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0cc15b9c0 con 0x7fc0cc06fb50 2024-09-13T16:51:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fd871a00700 1 -- 172.21.15.3:0/3096887657 shutdown_connections 2024-09-13T16:51:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fd871a00700 1 --2- 172.21.15.3:0/3096887657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c06bab0 0x7fd86c070bd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fd871a00700 1 --2- 172.21.15.3:0/3096887657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c0cca20 0x7fd86c070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fd871a00700 1 -- 172.21.15.3:0/3096887657 >> 172.21.15.3:0/3096887657 conn(0x7fd86c0cb620 msgr2=0x7fd86c0cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d4200700 1 -- 172.21.15.3:0/1441923877 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0cc15bf80 con 0x7fc0cc15b540 2024-09-13T16:51:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d4200700 1 -- 172.21.15.3:0/1441923877 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15bca0 con 0x7fc0cc0b5f70 2024-09-13T16:51:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f2821800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f281c1543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f2821800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281c0af130 0x7f281c156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f2821800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281c0afd30 0x7f281c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f2821800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c14dc70 con 0x7f2800003bb0 2024-09-13T16:51:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f2821800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c14d460 con 0x7f281c0afd30 2024-09-13T16:51:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f2821800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c151c90 con 0x7f281c0af130 2024-09-13T16:51:02.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0cbe00700 1 -- 172.21.15.3:0/1441923877 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749786572 0 0) 0x7fc0cc15b9c0 con 0x7fc0cc06fb50 2024-09-13T16:51:02.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fd871a00700 1 -- 172.21.15.3:0/3096887657 shutdown_connections 2024-09-13T16:51:02.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fd871a00700 1 -- 172.21.15.3:0/3096887657 wait complete. 2024-09-13T16:51:02.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f281a000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281c0af130 0x7f281c156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f281b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281c0afd30 0x7f281c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f281b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281c0afd30 0x7f281c156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52232/0 (socket says 172.21.15.3:52232) 2024-09-13T16:51:02.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f281b400700 1 -- 172.21.15.3:0/4239067238 learned_addr learned my addr 172.21.15.3:0/4239067238 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d0e00700 1 -- 172.21.15.3:0/1441923877 >> v2:172.21.15.157:3300/0 conn(0x7fc0cc15b540 msgr2=0x7fc0cc0cffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T16:51:02.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d0e00700 1 --2- 172.21.15.3:0/1441923877 >> v2:172.21.15.157:3300/0 conn(0x7fc0cc15b540 0x7fc0cc0cffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d0e00700 1 -- 172.21.15.3:0/1441923877 >> v1:172.21.15.3:6789/0 conn(0x7fc0cc06fb50 legacy=0x7fc0cc0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d0e00700 1 -- 172.21.15.3:0/1441923877 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc0b3e90 con 0x7fc0cc0b5f70 2024-09-13T16:51:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f281b400700 1 -- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281c0af130 msgr2=0x7f281c156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f281b400700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281c0af130 0x7f281c156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7fc0d0e00700 1 --2- 172.21.15.3:0/1441923877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc0b5f70 0x7fc0cc15b000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009850 tx=0x7fc0b8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=139bd8b26af050dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f281b400700 1 -- 172.21.15.3:0/4239067238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 msgr2=0x7f281c1543e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:51:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f2296c00700 1 Processor -- start 2024-09-13T16:51:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f2296c00700 1 -- start start 2024-09-13T16:51:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.925+0000 7f281aa00700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f281c1543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f281b400700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f281c1543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f281b400700 1 -- 172.21.15.3:0/4239067238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c150350 con 0x7f281c0afd30 2024-09-13T16:51:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f281a000700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281c0af130 0x7f281c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f281aa00700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f281c1543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871a00700 1 Processor -- start 2024-09-13T16:51:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f8993000700 1 Processor -- start 2024-09-13T16:51:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0cbe00700 1 -- 172.21.15.3:0/1441923877 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8011830 con 0x7fc0cc0b5f70 2024-09-13T16:51:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0cbe00700 1 -- 172.21.15.3:0/1441923877 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b8011990 con 0x7fc0cc0b5f70 2024-09-13T16:51:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0cbe00700 1 -- 172.21.15.3:0/1441923877 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8011d20 con 0x7fc0cc0b5f70 2024-09-13T16:51:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871a00700 1 -- start start 2024-09-13T16:51:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f8993000700 1 -- start start 2024-09-13T16:51:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2296c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2290071320 0x7f229015d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2296c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f229013e570 0x7f2290071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0d4200700 1 -- 172.21.15.3:0/1441923877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc0b5f70 msgr2=0x7fc0cc15b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0d4200700 1 --2- 172.21.15.3:0/1441923877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc0b5f70 0x7fc0cc15b000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009850 tx=0x7fc0b8009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f281b400700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281c0afd30 0x7f281c156850 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28000047f0 tx=0x7f28140114b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2812a00700 1 -- 172.21.15.3:0/4239067238 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281400e070 con 0x7f281c0afd30 2024-09-13T16:51:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2812a00700 1 -- 172.21.15.3:0/4239067238 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2814003c50 con 0x7f281c0afd30 2024-09-13T16:51:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2812a00700 1 -- 172.21.15.3:0/4239067238 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814007ee0 con 0x7f281c0afd30 2024-09-13T16:51:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c14d6b0 con 0x7f281c0afd30 2024-09-13T16:51:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f8ea7200700 1 Processor -- start 2024-09-13T16:51:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c06bab0 0x7fd86c1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c0cca20 0x7fd86c14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86c15a7c0 0x7fd86c149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c1502d0 con 0x7fd86c15a7c0 2024-09-13T16:51:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c150030 con 0x7fd86c0cca20 2024-09-13T16:51:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c155c40 con 0x7fd86c06bab0 2024-09-13T16:51:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2296c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2290071c40 con 0x7f229006e110 2024-09-13T16:51:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f228fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2290071320 0x7f229015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0d4200700 1 -- 172.21.15.3:0/1441923877 shutdown_connections 2024-09-13T16:51:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0d4200700 1 --2- 172.21.15.3:0/1441923877 >> v2:172.21.15.157:3300/0 conn(0x7fc0cc15b540 0x7fc0cc0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0d4200700 1 --2- 172.21.15.3:0/1441923877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc0b5f70 0x7fc0cc15b000 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0d4200700 1 -- 172.21.15.3:0/1441923877 >> 172.21.15.3:0/1441923877 conn(0x7fc0cc06a9c0 msgr2=0x7fc0cc0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0d4200700 1 -- 172.21.15.3:0/1441923877 shutdown_connections 2024-09-13T16:51:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fc0d4200700 1 -- 172.21.15.3:0/1441923877 wait complete. 2024-09-13T16:51:02.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c151920 con 0x7f281c0afd30 2024-09-13T16:51:02.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f8993000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898c0705b0 0x7f898c13ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f8993000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898c0b39e0 0x7f898c0c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f8993000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 0x7f898c15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2296c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22900b2df0 con 0x7f229013e570 2024-09-13T16:51:02.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f2296c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229015d870 con 0x7f2290071320 2024-09-13T16:51:02.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86c15a7c0 0x7fd86c149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd86be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c06bab0 0x7fd86c1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86c15a7c0 0x7fd86c149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54828/0 (socket says 172.21.15.3:54828) 2024-09-13T16:51:02.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7fd871000700 1 -- 172.21.15.3:0/1674224195 learned_addr learned my addr 172.21.15.3:0/1674224195 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f228fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2290071320 0x7f229015d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52246/0 (socket says 172.21.15.3:52246) 2024-09-13T16:51:02.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.929+0000 7f228fe00700 1 -- 172.21.15.3:0/2034754583 learned_addr learned my addr 172.21.15.3:0/2034754583 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898c0705b0 0x7f898c13ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd86b400700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c0cca20 0x7fd86c14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f228f400700 1 --2- 172.21.15.3:0/2034754583 >> v2:172.21.15.3:3300/0 conn(0x7f229013e570 0x7f2290071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898c0705b0 0x7f898c13ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52952/0 (socket says 172.21.15.3:52952) 2024-09-13T16:51:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8993000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c13d080 con 0x7f898c0b39e0 2024-09-13T16:51:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898c0b39e0 0x7f898c0c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8993000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c13cda0 con 0x7f898c141c20 2024-09-13T16:51:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8993000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0c3d80 con 0x7f898c0705b0 2024-09-13T16:51:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 -- 172.21.15.3:0/578190674 learned_addr learned my addr 172.21.15.3:0/578190674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8991000700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 0x7f898c15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd86b400700 1 -- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c06bab0 msgr2=0x7fd86c1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd86b400700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c06bab0 0x7fd86c1518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd86b400700 1 -- 172.21.15.3:0/1674224195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86c15a7c0 msgr2=0x7fd86c149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd86b400700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86c15a7c0 0x7fd86c149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f228fe00700 1 -- 172.21.15.3:0/2034754583 >> v2:172.21.15.3:3300/0 conn(0x7f229013e570 msgr2=0x7f2290071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fc0d4200700 1 Processor -- start 2024-09-13T16:51:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fc0d4200700 1 -- start start 2024-09-13T16:51:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 -- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 msgr2=0x7f898c15b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 0x7f898c15b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 -- 172.21.15.3:0/578190674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898c0b39e0 msgr2=0x7f898c0c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898c0b39e0 0x7f898c0c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 -- 172.21.15.3:0/578190674 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780094b0 con 0x7f898c0705b0 2024-09-13T16:51:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd86b400700 1 -- 172.21.15.3:0/1674224195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd864009a70 con 0x7fd86c0cca20 2024-09-13T16:51:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd86be00700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c06bab0 0x7fd86c1518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f228fe00700 1 --2- 172.21.15.3:0/2034754583 >> v2:172.21.15.3:3300/0 conn(0x7f229013e570 0x7f2290071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f228fe00700 1 -- 172.21.15.3:0/2034754583 >> v1:172.21.15.3:6789/0 conn(0x7f229006e110 legacy=0x7f22900cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f228fe00700 1 -- 172.21.15.3:0/2034754583 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900b4250 con 0x7f2290071320 2024-09-13T16:51:02.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd86b400700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c0cca20 0x7fd86c14faf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0cc190 tx=0x7fd85800caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f2812a00700 1 -- 172.21.15.3:0/4239067238 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2814011a40 con 0x7f281c0afd30 2024-09-13T16:51:02.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f898be00700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898c0705b0 0x7f898c13ee20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8978004000 tx=0x7f8978004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd863e00700 1 -- 172.21.15.3:0/1674224195 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858011800 con 0x7fd86c0cca20 2024-09-13T16:51:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd863e00700 1 -- 172.21.15.3:0/1674224195 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858011960 con 0x7fd86c0cca20 2024-09-13T16:51:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd863e00700 1 -- 172.21.15.3:0/1674224195 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858011cd0 con 0x7fd86c0cca20 2024-09-13T16:51:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c151780 con 0x7f281c0afd30 2024-09-13T16:51:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f2812a00700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f27fc066830 0x7f27fc068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f2812a00700 1 -- 172.21.15.3:0/4239067238 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2814084a10 con 0x7f281c0afd30 2024-09-13T16:51:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8ea7200700 1 -- start start 2024-09-13T16:51:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8ea7200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ea8146ee0 0x7f8ea81472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd871a00700 1 -- 172.21.15.3:0/1674224195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c155f20 con 0x7fd86c0cca20 2024-09-13T16:51:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7fd871a00700 1 -- 172.21.15.3:0/1674224195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86c1564e0 con 0x7fd86c0cca20 2024-09-13T16:51:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f228fe00700 1 --2- 172.21.15.3:0/2034754583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290071320 0x7f229015d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f229006f490 tx=0x7f2280009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=343d34f6ebcc6d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8983e00700 1 -- 172.21.15.3:0/578190674 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978003d20 con 0x7f898c0705b0 2024-09-13T16:51:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8983e00700 1 -- 172.21.15.3:0/578190674 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897801f070 con 0x7f898c0705b0 2024-09-13T16:51:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.933+0000 7f8983e00700 1 -- 172.21.15.3:0/578190674 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978003ea0 con 0x7f898c0705b0 2024-09-13T16:51:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c0c4060 con 0x7f898c0705b0 2024-09-13T16:51:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea7200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ea80705b0 0x7f8ea806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea7200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea8165800 con 0x7f8ea806c5e0 2024-09-13T16:51:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea7200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea8070a80 con 0x7f8ea8146ee0 2024-09-13T16:51:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea7200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea81626e0 con 0x7f8ea80705b0 2024-09-13T16:51:02.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c13f360 con 0x7f898c0705b0 2024-09-13T16:51:02.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea5e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ea8146ee0 0x7f8ea81472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f281aa00700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f27fc066830 0x7f27fc068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea5e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ea8146ee0 0x7f8ea81472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54848/0 (socket says 172.21.15.3:54848) 2024-09-13T16:51:02.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea5e00700 1 -- 172.21.15.3:0/3026495110 learned_addr learned my addr 172.21.15.3:0/3026495110 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea6800700 1 --2- 172.21.15.3:0/3026495110 >> v2:172.21.15.157:3300/0 conn(0x7f8ea80705b0 0x7f8ea806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d4200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc06fb50 0x7fc0cc15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea5e00700 1 -- 172.21.15.3:0/3026495110 >> v2:172.21.15.157:3300/0 conn(0x7f8ea80705b0 msgr2=0x7f8ea806ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:02.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea5e00700 1 --2- 172.21.15.3:0/3026495110 >> v2:172.21.15.157:3300/0 conn(0x7f8ea80705b0 0x7f8ea806ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea5e00700 1 -- 172.21.15.3:0/3026495110 >> v1:172.21.15.157:6789/0 conn(0x7f8ea806c5e0 legacy=0x7f8ea806fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c0c4580 con 0x7f898c0705b0 2024-09-13T16:51:02.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d4200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0cc0b5f70 0x7fc0cc150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fd861600700 1 -- 172.21.15.3:0/1674224195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86c156700 con 0x7fd86c0cca20 2024-09-13T16:51:02.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea5e00700 1 -- 172.21.15.3:0/3026495110 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8164ed0 con 0x7f8ea8146ee0 2024-09-13T16:51:02.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d4200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0cc15b540 0x7fc0cc1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fd863e00700 1 -- 172.21.15.3:0/1674224195 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd858022350 con 0x7fd86c0cca20 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d2200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0cc15b540 0x7fc0cc1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d0e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0cc0b5f70 0x7fc0cc150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d0e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0cc0b5f70 0x7fc0cc150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52282/0 (socket says 172.21.15.3:52282) 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d0e00700 1 -- 172.21.15.3:0/668578381 learned_addr learned my addr 172.21.15.3:0/668578381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15ac00 con 0x7fc0cc15b540 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc1f3700 con 0x7fc0cc0b5f70 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc1f39e0 con 0x7fc0cc06fb50 2024-09-13T16:51:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d1800700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc06fb50 0x7fc0cc15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f281aa00700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f27fc066830 0x7f27fc068cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f280c015040 tx=0x7f280c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fd863e00700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd84c066780 0x7fd84c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7f8ea5e00700 1 --2- 172.21.15.3:0/3026495110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 0x7f8ea81472c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8164bf0 tx=0x7f8e98009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b795ac60287bfee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fd863e00700 1 -- 172.21.15.3:0/1674224195 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd858084210 con 0x7fd86c0cca20 2024-09-13T16:51:02.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea4a00700 1 -- 172.21.15.3:0/3026495110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800ba20 con 0x7f8ea8146ee0 2024-09-13T16:51:02.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.937+0000 7fc0d2200700 1 -- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc06fb50 msgr2=0x7fc0cc15a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0d2200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc06fb50 0x7fc0cc15a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0d2200700 1 -- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0cc0b5f70 msgr2=0x7fc0cc150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0d2200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0cc0b5f70 0x7fc0cc150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0d2200700 1 -- 172.21.15.3:0/668578381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80094b0 con 0x7fc0cc15b540 2024-09-13T16:51:02.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0d1800700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc06fb50 0x7fc0cc15a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:02.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e30a00700 1 Processor -- start 2024-09-13T16:51:02.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea4a00700 1 -- 172.21.15.3:0/3026495110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800bb80 con 0x7f8ea8146ee0 2024-09-13T16:51:02.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e30a00700 1 -- start start 2024-09-13T16:51:02.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea4a00700 1 -- 172.21.15.3:0/3026495110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98009e60 con 0x7f8ea8146ee0 2024-09-13T16:51:02.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fd86be00700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd84c066780 0x7fd84c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0d2200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0cc15b540 0x7fc0cc1588b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400be20 tx=0x7fc0c4009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8983e00700 1 -- 172.21.15.3:0/578190674 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8978021030 con 0x7f898c0705b0 2024-09-13T16:51:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea7200700 1 -- 172.21.15.3:0/3026495110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 msgr2=0x7f8ea81472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea7200700 1 --2- 172.21.15.3:0/3026495110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 0x7f8ea81472c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8164bf0 tx=0x7f8e98009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea7200700 1 -- 172.21.15.3:0/3026495110 shutdown_connections 2024-09-13T16:51:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea7200700 1 --2- 172.21.15.3:0/3026495110 >> v2:172.21.15.157:3300/0 conn(0x7f8ea80705b0 0x7f8ea806ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea7200700 1 --2- 172.21.15.3:0/3026495110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 0x7f8ea81472c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea7200700 1 -- 172.21.15.3:0/3026495110 >> 172.21.15.3:0/3026495110 conn(0x7f8ea80b0cf0 msgr2=0x7f8ea80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea7200700 1 -- 172.21.15.3:0/3026495110 shutdown_connections 2024-09-13T16:51:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8ea7200700 1 -- 172.21.15.3:0/3026495110 wait complete. 2024-09-13T16:51:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8983e00700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f896c066800 0x7f896c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f8983e00700 1 -- 172.21.15.3:0/578190674 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8978085ab0 con 0x7f898c0705b0 2024-09-13T16:51:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fd86be00700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd84c066780 0x7fd84c068c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd85c007fd0 tx=0x7fd85c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0c9600700 1 -- 172.21.15.3:0/668578381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400cc90 con 0x7fc0cc15b540 2024-09-13T16:51:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f898b400700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f896c066800 0x7f896c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0cc1f3cc0 con 0x7fc0cc15b540 2024-09-13T16:51:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0c9600700 1 -- 172.21.15.3:0/668578381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c400ce30 con 0x7fc0cc15b540 2024-09-13T16:51:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e30a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e2c15cb10 0x7f1e2c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e30a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2c0add70 con 0x7f1e2c0af320 2024-09-13T16:51:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e30a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2c0b2f20 con 0x7f1e2c0ae1f0 2024-09-13T16:51:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e30a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e2c0ad4d0 con 0x7f1e2c15cb10 2024-09-13T16:51:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e29e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1e2c0ae1f0 0x7f1e2c0aec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37668/0 (socket says 172.21.15.3:37668) 2024-09-13T16:51:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e29e00700 1 -- 172.21.15.3:0/100972334 learned_addr learned my addr 172.21.15.3:0/100972334 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f898b400700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f896c066800 0x7f896c068cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f898c1f7f60 tx=0x7f8984001a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0c9600700 1 -- 172.21.15.3:0/668578381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4014680 con 0x7fc0cc15b540 2024-09-13T16:51:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc1f3360 con 0x7fc0cc15b540 2024-09-13T16:51:02.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.941+0000 7f1e2a800700 1 --2- 172.21.15.3:0/100972334 >> v2:172.21.15.3:3300/0 conn(0x7f1e2c15cb10 0x7f1e2c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f1e2a800700 1 -- 172.21.15.3:0/100972334 >> v1:172.21.15.157:6789/0 conn(0x7f1e2c0ae1f0 legacy=0x7f1e2c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f1e2a800700 1 -- 172.21.15.3:0/100972334 >> v1:172.21.15.3:6789/0 conn(0x7f1e2c0af320 legacy=0x7f1e2c0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f1e2a800700 1 -- 172.21.15.3:0/100972334 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0ad7b0 con 0x7f1e2c15cb10 2024-09-13T16:51:02.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0cc155db0 con 0x7fc0cc15b540 2024-09-13T16:51:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f1e2a800700 1 --2- 172.21.15.3:0/100972334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 0x7f1e2c0b4e70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009850 tx=0x7f1e1c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbd94b5f1e7d665 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea7200700 1 Processor -- start 2024-09-13T16:51:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f228ea00700 1 -- 172.21.15.3:0/2034754583 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000b8a0 con 0x7f2290071320 2024-09-13T16:51:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f2812a00700 1 -- 172.21.15.3:0/4239067238 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f281c151780 con 0x7f281c0afd30 2024-09-13T16:51:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea7200700 1 -- start start 2024-09-13T16:51:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f1e29400700 1 -- 172.21.15.3:0/100972334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c011830 con 0x7f1e2c15cb10 2024-09-13T16:51:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7fc0c9600700 1 -- 172.21.15.3:0/668578381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc0c40147e0 con 0x7fc0cc15b540 2024-09-13T16:51:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea7200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ea806c5e0 0x7f8ea81f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea80705b0 0x7f8ea8143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea7200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 0x7f8ea8161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea7200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81ebc30 con 0x7f8ea8146ee0 2024-09-13T16:51:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81f7000 con 0x7f8ea80705b0 2024-09-13T16:51:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea7200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81f72e0 con 0x7f8ea806c5e0 2024-09-13T16:51:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.945+0000 7f8ea5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea80705b0 0x7f8ea8143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea80705b0 0x7f8ea8143d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52298/0 (socket says 172.21.15.3:52298) 2024-09-13T16:51:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5400700 1 -- 172.21.15.3:0/1435377974 learned_addr learned my addr 172.21.15.3:0/1435377974 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea6800700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 0x7f8ea8161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5e00700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ea806c5e0 0x7f8ea81f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8983e00700 1 -- 172.21.15.3:0/578190674 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8978052ba0 con 0x7f898c0705b0 2024-09-13T16:51:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7fc0c9600700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc0ac066730 0x7fc0ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5400700 1 -- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ea806c5e0 msgr2=0x7f8ea81f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5400700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ea806c5e0 0x7f8ea81f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5400700 1 -- 172.21.15.3:0/1435377974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 msgr2=0x7f8ea8161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5400700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 0x7f8ea8161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5400700 1 -- 172.21.15.3:0/1435377974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98009570 con 0x7f8ea80705b0 2024-09-13T16:51:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea6800700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 0x7f8ea8161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7fc0c9600700 1 -- 172.21.15.3:0/668578381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc0c4084840 con 0x7fc0cc15b540 2024-09-13T16:51:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7fc0d1800700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc0ac066730 0x7fc0ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f8ea5400700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea80705b0 0x7f8ea8143d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400c500 tx=0x7f8e9400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7fc0d1800700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc0ac066730 0x7fc0ac068bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00018e0 tx=0x7fc0c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f228ea00700 1 -- 172.21.15.3:0/2034754583 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228000ba00 con 0x7f2290071320 2024-09-13T16:51:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f228ea00700 1 -- 172.21.15.3:0/2034754583 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000bd90 con 0x7f2290071320 2024-09-13T16:51:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7fd863e00700 1 -- 172.21.15.3:0/1674224195 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd858051300 con 0x7fd86c0cca20 2024-09-13T16:51:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f2296c00700 1 -- 172.21.15.3:0/2034754583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290071320 msgr2=0x7f229015d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.949+0000 7f2296c00700 1 --2- 172.21.15.3:0/2034754583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290071320 0x7f229015d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f229006f490 tx=0x7f2280009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f2296c00700 1 -- 172.21.15.3:0/2034754583 shutdown_connections 2024-09-13T16:51:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f2296c00700 1 --2- 172.21.15.3:0/2034754583 >> v2:172.21.15.3:3300/0 conn(0x7f229013e570 0x7f2290071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f2296c00700 1 --2- 172.21.15.3:0/2034754583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290071320 0x7f229015d330 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f229006f490 tx=0x7f2280009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f2296c00700 1 -- 172.21.15.3:0/2034754583 >> 172.21.15.3:0/2034754583 conn(0x7f229006a390 msgr2=0x7f229006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f8e9e000700 1 -- 172.21.15.3:0/1435377974 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e940052d0 con 0x7f8ea80705b0 2024-09-13T16:51:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f8e9e000700 1 -- 172.21.15.3:0/1435377974 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e94005430 con 0x7f8ea80705b0 2024-09-13T16:51:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f8e9e000700 1 -- 172.21.15.3:0/1435377974 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e940055a0 con 0x7f8ea80705b0 2024-09-13T16:51:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea81f75c0 con 0x7f8ea80705b0 2024-09-13T16:51:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea81623d0 con 0x7f8ea80705b0 2024-09-13T16:51:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea81ebe30 con 0x7f8ea80705b0 2024-09-13T16:51:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f2296c00700 1 -- 172.21.15.3:0/2034754583 shutdown_connections 2024-09-13T16:51:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f1e29400700 1 -- 172.21.15.3:0/100972334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c011990 con 0x7f1e2c15cb10 2024-09-13T16:51:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f1e29400700 1 -- 172.21.15.3:0/100972334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c011d20 con 0x7f1e2c15cb10 2024-09-13T16:51:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f1e30a00700 1 -- 172.21.15.3:0/100972334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 msgr2=0x7f1e2c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f1e30a00700 1 --2- 172.21.15.3:0/100972334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 0x7f1e2c0b4e70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009850 tx=0x7f1e1c009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.953+0000 7f2296c00700 1 -- 172.21.15.3:0/2034754583 wait complete. 2024-09-13T16:51:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f8e9e000700 1 -- 172.21.15.3:0/1435377974 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8e940038e0 con 0x7f8ea80705b0 2024-09-13T16:51:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7fc0c9600700 1 -- 172.21.15.3:0/668578381 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc0c40519e0 con 0x7fc0cc15b540 2024-09-13T16:51:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f1e30a00700 1 -- 172.21.15.3:0/100972334 shutdown_connections 2024-09-13T16:51:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f1e30a00700 1 --2- 172.21.15.3:0/100972334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 0x7f1e2c0b4e70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f2296c00700 1 Processor -- start 2024-09-13T16:51:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f1e30a00700 1 -- 172.21.15.3:0/100972334 >> 172.21.15.3:0/100972334 conn(0x7f1e2c0710b0 msgr2=0x7f1e2c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f1e30a00700 1 -- 172.21.15.3:0/100972334 shutdown_connections 2024-09-13T16:51:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f2296c00700 1 -- start start 2024-09-13T16:51:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f1e30a00700 1 -- 172.21.15.3:0/100972334 wait complete. 2024-09-13T16:51:02.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f2296c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f229006e110 0x7f2290153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f2296c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229013e570 0x7f22901562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f229006e110 0x7f2290153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f229006e110 0x7f2290153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52978/0 (socket says 172.21.15.3:52978) 2024-09-13T16:51:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f1e30a00700 1 Processor -- start 2024-09-13T16:51:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f1e30a00700 1 -- start start 2024-09-13T16:51:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f2296c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f229013b060 0x7f229013a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290153ea0 con 0x7f229013e570 2024-09-13T16:51:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 -- 172.21.15.3:0/2806262721 learned_addr learned my addr 172.21.15.3:0/2806262721 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22901593a0 con 0x7f229013b060 2024-09-13T16:51:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2290163b90 con 0x7f229006e110 2024-09-13T16:51:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228f400700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229013e570 0x7f22901562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f2294c00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f229013b060 0x7f229013a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 -- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f229013b060 msgr2=0x7f229013a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f229013b060 0x7f229013a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 -- 172.21.15.3:0/2806262721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229013e570 msgr2=0x7f22901562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f8e9e000700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e88066780 0x7f8e88068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229013e570 0x7f22901562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 -- 172.21.15.3:0/2806262721 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2280009570 con 0x7f229006e110 2024-09-13T16:51:02.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.957+0000 7f8e9e000700 1 -- 172.21.15.3:0/1435377974 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8e94083b50 con 0x7f8ea80705b0 2024-09-13T16:51:02.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f8ea5e00700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e88066780 0x7f8e88068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228f400700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229013e570 0x7f22901562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:02.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f228fe00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f229006e110 0x7f2290153960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2280013000 tx=0x7f2280004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f8e9e000700 1 -- 172.21.15.3:0/1435377974 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8e94050bb0 con 0x7f8ea80705b0 2024-09-13T16:51:02.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f1e30a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c0ae1f0 0x7f1e2c153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.961+0000 7f1e30a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e2c0af320 0x7f1e2c0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c0ae1f0 0x7f1e2c153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f2287e00700 1 -- 172.21.15.3:0/2806262721 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280004450 con 0x7f229006e110 2024-09-13T16:51:02.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f2287e00700 1 -- 172.21.15.3:0/2806262721 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22800045b0 con 0x7f229006e110 2024-09-13T16:51:02.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f2287e00700 1 -- 172.21.15.3:0/2806262721 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280003900 con 0x7f229006e110 2024-09-13T16:51:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f8ea5e00700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e88066780 0x7f8e88068c40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009890 tx=0x7f8e98008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c0ae1f0 0x7f1e2c153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52320/0 (socket says 172.21.15.3:52320) 2024-09-13T16:51:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e29e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e2c0af320 0x7f1e2c0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e29e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e2c0af320 0x7f1e2c0b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:52992/0 (socket says 172.21.15.3:52992) 2024-09-13T16:51:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e30a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 0x7f1e2c15b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e30a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1f3390 con 0x7f1e2c15cb10 2024-09-13T16:51:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e30a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1f2490 con 0x7f1e2c0ae1f0 2024-09-13T16:51:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e30a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1f2710 con 0x7f1e2c0af320 2024-09-13T16:51:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2a800700 1 -- 172.21.15.3:0/2548693503 learned_addr learned my addr 172.21.15.3:0/2548693503 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2b200700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 0x7f1e2c15b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2a800700 1 -- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e2c0af320 msgr2=0x7f1e2c0b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2a800700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e2c0af320 0x7f1e2c0b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2a800700 1 -- 172.21.15.3:0/2548693503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 msgr2=0x7f1e2c15b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2a800700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 0x7f1e2c15b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2a800700 1 -- 172.21.15.3:0/2548693503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c0094b0 con 0x7f1e2c0ae1f0 2024-09-13T16:51:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e2b200700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 0x7f1e2c15b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f1e29e00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e2c0af320 0x7f1e2c0b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-13T16:51:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2290163e70 con 0x7f229006e110 2024-09-13T16:51:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.965+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290154370 con 0x7f229006e110 2024-09-13T16:51:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.969+0000 7f1e2a800700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c0ae1f0 0x7f1e2c153090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0b2b70 tx=0x7f1e1c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.969+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229006ca00 con 0x7f229006e110 2024-09-13T16:51:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.969+0000 7f1e22a00700 1 -- 172.21.15.3:0/2548693503 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0038c0 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.969+0000 7f1e22a00700 1 -- 172.21.15.3:0/2548693503 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c003a20 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.969+0000 7f1e22a00700 1 -- 172.21.15.3:0/2548693503 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c003dd0 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.969+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2c1f29f0 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.969+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c1f2fb0 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.969+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2c0ad210 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.973+0000 7f2287e00700 1 -- 172.21.15.3:0/2806262721 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2280003a60 con 0x7f229006e110 2024-09-13T16:51:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.973+0000 7f1e22a00700 1 -- 172.21.15.3:0/2548693503 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e1c030480 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.973+0000 7f1e22a00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1e0c066800 0x7f1e0c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.973+0000 7f2287e00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f226c066800 0x7f226c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.973+0000 7f2287e00700 1 -- 172.21.15.3:0/2806262721 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f228008e8a0 con 0x7f229006e110 2024-09-13T16:51:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.973+0000 7f1e22a00700 1 -- 172.21.15.3:0/2548693503 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1e1c021030 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.977+0000 7f1e29e00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1e0c066800 0x7f1e0c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.977+0000 7f228f400700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f226c066800 0x7f226c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.977+0000 7f1e29e00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1e0c066800 0x7f1e0c068cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0b0bd0 tx=0x7f1e18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.977+0000 7f2287e00700 1 -- 172.21.15.3:0/2806262721 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f228005b990 con 0x7f229006e110 2024-09-13T16:51:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.977+0000 7f228f400700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f226c066800 0x7f226c068cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f22901f6050 tx=0x7f2288003a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:02.981+0000 7f1e22a00700 1 -- 172.21.15.3:0/2548693503 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1e1c053360 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.249+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f85fc0d0ad0 con 0x7f85fc158410 2024-09-13T16:51:03.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.249+0000 7f85f8c00700 1 -- 172.21.15.3:0/2743039224 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f85f402c090 con 0x7f85fc158410 2024-09-13T16:51:03.256 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2024-09-13T16:51:03.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.253+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f85dc066730 msgr2=0x7f85dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.253+0000 7f8603c00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f85dc066730 0x7f85dc068bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f85f0000f60 tx=0x7f85f0009040 comp rx=0 tx=0).stop 2024-09-13T16:51:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.253+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc158410 msgr2=0x7f85fc1547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.253+0000 7f8603c00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc158410 0x7f85fc1547b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85f4014040 tx=0x7f85f4004b20 comp rx=0 tx=0).stop 2024-09-13T16:51:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.261+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 shutdown_connections 2024-09-13T16:51:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.261+0000 7f8603c00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f85dc066730 0x7f85dc068bf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.261+0000 7f8603c00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85fc158410 0x7f85fc1547b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.261+0000 7f8603c00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc070260 0x7f85fc154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.261+0000 7f8603c00700 1 --2- 172.21.15.3:0/2743039224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85fc06fdf0 0x7f85fc203230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.261+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 >> 172.21.15.3:0/2743039224 conn(0x7f85fc0b2a90 msgr2=0x7f85fc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.261+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 shutdown_connections 2024-09-13T16:51:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.261+0000 7f8603c00700 1 -- 172.21.15.3:0/2743039224 wait complete. 2024-09-13T16:51:03.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-09-13T16:51:03.293 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:03.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.293+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f281c151780 con 0x7f281c0afd30 2024-09-13T16:51:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.293+0000 7f2812a00700 1 -- 172.21.15.3:0/4239067238 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f281c151780 con 0x7f281c0afd30 2024-09-13T16:51:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.293+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f898c0c3ec0 con 0x7f898c0705b0 2024-09-13T16:51:03.298 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2024-09-13T16:51:03.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f27fc066830 msgr2=0x7f27fc068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f27fc066830 0x7f27fc068cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f280c015040 tx=0x7f280c007000 comp rx=0 tx=0).stop 2024-09-13T16:51:03.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281c0afd30 msgr2=0x7f281c156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f8983e00700 1 -- 172.21.15.3:0/578190674 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f898c0c3ec0 con 0x7f898c0705b0 2024-09-13T16:51:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281c0afd30 0x7f281c156850 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28000047f0 tx=0x7f28140114b0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 shutdown_connections 2024-09-13T16:51:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f27fc066830 0x7f27fc068cf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281c0afd30 0x7f281c156850 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281c0af130 0x7f281c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 --2- 172.21.15.3:0/4239067238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2800003bb0 0x7f281c1543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 >> 172.21.15.3:0/4239067238 conn(0x7f281c06b1e0 msgr2=0x7f281c15af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 shutdown_connections 2024-09-13T16:51:03.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.297+0000 7f2821800700 1 -- 172.21.15.3:0/4239067238 wait complete. 2024-09-13T16:51:03.304 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:51:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7fd861600700 1 -- 172.21.15.3:0/1674224195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd86c06b3a0 con 0x7fd86c0cca20 2024-09-13T16:51:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f896c066800 msgr2=0x7f896c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f896c066800 0x7f896c068cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f898c1f7f60 tx=0x7f8984001a60 comp rx=0 tx=0).stop 2024-09-13T16:51:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898c0705b0 msgr2=0x7f898c13ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc0cc155db0 con 0x7fc0cc15b540 2024-09-13T16:51:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898c0705b0 0x7f898c13ee20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8978004000 tx=0x7f8978004770 comp rx=0 tx=0).stop 2024-09-13T16:51:03.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 shutdown_connections 2024-09-13T16:51:03.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.305+0000 7fd863e00700 1 -- 172.21.15.3:0/1674224195 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd858054920 con 0x7fd86c0cca20 2024-09-13T16:51:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f896c066800 0x7f896c068cc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c141c20 0x7f898c15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898c0b39e0 0x7f898c0c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 --2- 172.21.15.3:0/578190674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898c0705b0 0x7f898c13ee20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 >> 172.21.15.3:0/578190674 conn(0x7f898c06b1e0 msgr2=0x7f898c0ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.301+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 shutdown_connections 2024-09-13T16:51:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.305+0000 7f8993000700 1 -- 172.21.15.3:0/578190674 wait complete. 2024-09-13T16:51:03.311 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:51:03.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.305+0000 7fc0c9600700 1 -- 172.21.15.3:0/668578381 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc0c4023020 con 0x7fc0cc15b540 2024-09-13T16:51:03.312 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:51:03.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.305+0000 7fd861600700 1 -- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd84c066780 msgr2=0x7fd84c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.305+0000 7fd861600700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd84c066780 0x7fd84c068c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd85c007fd0 tx=0x7fd85c007de0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.305+0000 7fd861600700 1 -- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c0cca20 msgr2=0x7fd86c14faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.305+0000 7fd861600700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c0cca20 0x7fd86c14faf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0cc190 tx=0x7fd85800caa0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2290072e90 con 0x7f229006e110 2024-09-13T16:51:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fd861600700 1 -- 172.21.15.3:0/1674224195 shutdown_connections 2024-09-13T16:51:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fd861600700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd84c066780 0x7fd84c068c40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fd861600700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86c15a7c0 0x7fd86c149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fd861600700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd86c0cca20 0x7fd86c14faf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fd861600700 1 --2- 172.21.15.3:0/1674224195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86c06bab0 0x7fd86c1518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc0ac066730 msgr2=0x7fc0ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc0ac066730 0x7fc0ac068bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00018e0 tx=0x7fc0c0009040 comp rx=0 tx=0).stop 2024-09-13T16:51:03.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0cc15b540 msgr2=0x7fc0cc1588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0cc15b540 0x7fc0cc1588b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400be20 tx=0x7fc0c4009ef0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fd861600700 1 -- 172.21.15.3:0/1674224195 >> 172.21.15.3:0/1674224195 conn(0x7fd86c0cb620 msgr2=0x7fd86c070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fd861600700 1 -- 172.21.15.3:0/1674224195 shutdown_connections 2024-09-13T16:51:03.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fd861600700 1 -- 172.21.15.3:0/1674224195 wait complete. 2024-09-13T16:51:03.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7f2287e00700 1 -- 172.21.15.3:0/2806262721 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f228005efb0 con 0x7f229006e110 2024-09-13T16:51:03.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 shutdown_connections 2024-09-13T16:51:03.317 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:51:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc0ac066730 0x7fc0ac068bf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0cc15b540 0x7fc0cc1588b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0cc0b5f70 0x7fc0cc150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 --2- 172.21.15.3:0/668578381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0cc06fb50 0x7fc0cc15a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 >> 172.21.15.3:0/668578381 conn(0x7fc0cc06a9c0 msgr2=0x7fc0cc0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8ea8067ce0 con 0x7f8ea80705b0 2024-09-13T16:51:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 shutdown_connections 2024-09-13T16:51:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7fc0d4200700 1 -- 172.21.15.3:0/668578381 wait complete. 2024-09-13T16:51:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.309+0000 7f8e9e000700 1 -- 172.21.15.3:0/1435377974 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e94017020 con 0x7f8ea80705b0 2024-09-13T16:51:03.319 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2024-09-13T16:51:03.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.313+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f226c066800 msgr2=0x7f226c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.313+0000 7f2296c00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f226c066800 0x7f226c068cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f22901f6050 tx=0x7f2288003a80 comp rx=0 tx=0).stop 2024-09-13T16:51:03.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.313+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f229006e110 msgr2=0x7f2290153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.313+0000 7f2296c00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f229006e110 0x7f2290153960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2280013000 tx=0x7f2280004770 comp rx=0 tx=0).stop 2024-09-13T16:51:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 shutdown_connections 2024-09-13T16:51:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f2296c00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f226c066800 0x7f226c068cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f2296c00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f229013b060 0x7f229013a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f2296c00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229013e570 0x7f22901562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f2296c00700 1 --2- 172.21.15.3:0/2806262721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f229006e110 0x7f2290153960 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 >> 172.21.15.3:0/2806262721 conn(0x7f229006a390 msgr2=0x7f22900ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 shutdown_connections 2024-09-13T16:51:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e88066780 msgr2=0x7f8e88068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e88066780 0x7f8e88068c40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009890 tx=0x7f8e98008030 comp rx=0 tx=0).stop 2024-09-13T16:51:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea80705b0 msgr2=0x7f8ea8143d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea80705b0 0x7f8ea8143d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400c500 tx=0x7f8e9400c810 comp rx=0 tx=0).stop 2024-09-13T16:51:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f2296c00700 1 -- 172.21.15.3:0/2806262721 wait complete. 2024-09-13T16:51:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 shutdown_connections 2024-09-13T16:51:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e88066780 0x7f8e88068c40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ea8146ee0 0x7f8ea8161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea80705b0 0x7f8ea8143d70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 --2- 172.21.15.3:0/1435377974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ea806c5e0 0x7f8ea81f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 >> 172.21.15.3:0/1435377974 conn(0x7f8ea80b0cf0 msgr2=0x7f8ea80cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 shutdown_connections 2024-09-13T16:51:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.317+0000 7f8ea7200700 1 -- 172.21.15.3:0/1435377974 wait complete. 2024-09-13T16:51:03.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.329+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1e2c0b4e70 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.329+0000 7f1e22a00700 1 -- 172.21.15.3:0/2548693503 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e1c02b020 con 0x7f1e2c0ae1f0 2024-09-13T16:51:03.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-09-13T16:51:03.336 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:03.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-09-13T16:51:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2024-09-13T16:51:03.337 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:03.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-09-13T16:51:03.339 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1e0c066800 msgr2=0x7f1e0c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1e0c066800 0x7f1e0c068cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0b0bd0 tx=0x7f1e18009040 comp rx=0 tx=0).stop 2024-09-13T16:51:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c0ae1f0 msgr2=0x7f1e2c153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c0ae1f0 0x7f1e2c153090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0b2b70 tx=0x7f1e1c004770 comp rx=0 tx=0).stop 2024-09-13T16:51:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 shutdown_connections 2024-09-13T16:51:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1e0c066800 0x7f1e0c068cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e2c15cb10 0x7f1e2c15b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e2c0af320 0x7f1e2c0b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 --2- 172.21.15.3:0/2548693503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e2c0ae1f0 0x7f1e2c153090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 >> 172.21.15.3:0/2548693503 conn(0x7f1e2c0710b0 msgr2=0x7f1e2c0b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 shutdown_connections 2024-09-13T16:51:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.333+0000 7f1e30a00700 1 -- 172.21.15.3:0/2548693503 wait complete. 2024-09-13T16:51:03.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.0 2024-09-13T16:51:03.342 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:03.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.3 2024-09-13T16:51:03.343 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:03.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.2 2024-09-13T16:51:03.345 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:03.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.5 2024-09-13T16:51:03.361 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:03.362 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T16:51:03.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:51:03.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.561+0000 7f796d200700 1 Processor -- start 2024-09-13T16:51:03.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.561+0000 7f796d200700 1 -- start start 2024-09-13T16:51:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f796d200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79681575e0 0x7f79680cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f796d200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 0x7f79680cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f796d200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79680cefd0 con 0x7f796813fdd0 2024-09-13T16:51:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f796d200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680cfe30 con 0x7f79681575e0 2024-09-13T16:51:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f796d200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796813d0d0 con 0x7f79680cf460 2024-09-13T16:51:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f7965e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79681575e0 0x7f79680cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f7965e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79681575e0 0x7f79680cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54888/0 (socket says 172.21.15.3:54888) 2024-09-13T16:51:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f7965e00700 1 -- 172.21.15.3:0/2923958470 learned_addr learned my addr 172.21.15.3:0/2923958470 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f7965e00700 1 -- 172.21.15.3:0/2923958470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 msgr2=0x7f79680cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f7965e00700 1 --2- 172.21.15.3:0/2923958470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 0x7f79680cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f7965e00700 1 -- 172.21.15.3:0/2923958470 >> v1:172.21.15.3:6789/0 conn(0x7f796813fdd0 legacy=0x7f796806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:03.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f7965e00700 1 -- 172.21.15.3:0/2923958470 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680ccfd0 con 0x7f79681575e0 2024-09-13T16:51:03.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.565+0000 7f7965e00700 1 --2- 172.21.15.3:0/2923958470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f79680cc9a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7958009910 tx=0x7f7958009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ae9d00f6d44cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f7965400700 1 -- 172.21.15.3:0/2923958470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800ba60 con 0x7f79681575e0 2024-09-13T16:51:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f7965400700 1 -- 172.21.15.3:0/2923958470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795800bbc0 con 0x7f79681575e0 2024-09-13T16:51:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f7965400700 1 -- 172.21.15.3:0/2923958470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958009e60 con 0x7f79681575e0 2024-09-13T16:51:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 -- 172.21.15.3:0/2923958470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 msgr2=0x7f79680cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 --2- 172.21.15.3:0/2923958470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f79680cc9a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7958009910 tx=0x7f7958009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 -- 172.21.15.3:0/2923958470 shutdown_connections 2024-09-13T16:51:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 --2- 172.21.15.3:0/2923958470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 0x7f79680cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 --2- 172.21.15.3:0/2923958470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f79680cc9a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 -- 172.21.15.3:0/2923958470 >> 172.21.15.3:0/2923958470 conn(0x7f796806d3c0 msgr2=0x7f796806d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 -- 172.21.15.3:0/2923958470 shutdown_connections 2024-09-13T16:51:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 -- 172.21.15.3:0/2923958470 wait complete. 2024-09-13T16:51:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 Processor -- start 2024-09-13T16:51:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.569+0000 7f796d200700 1 -- start start 2024-09-13T16:51:03.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f796d200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 0x7f79681f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f796d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796813fdd0 0x7f796806a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f796d200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f7968142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f796d200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796806b520 con 0x7f79681575e0 2024-09-13T16:51:03.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f796d200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796806b7c0 con 0x7f796813fdd0 2024-09-13T16:51:03.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f796d200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796806baa0 con 0x7f79680cf460 2024-09-13T16:51:03.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7966800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f7968142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7965e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 0x7f79681f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7965e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 0x7f79681f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:53006/0 (socket says 172.21.15.3:53006) 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7965e00700 1 -- 172.21.15.3:0/3216045439 learned_addr learned my addr 172.21.15.3:0/3216045439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7966800700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f7968142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54896/0 (socket says 172.21.15.3:54896) 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f795fe00700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796813fdd0 0x7f796806a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7966800700 1 -- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 msgr2=0x7f79681f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7966800700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 0x7f79681f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7966800700 1 -- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796813fdd0 msgr2=0x7f796806a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7966800700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796813fdd0 0x7f796806a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7966800700 1 -- 172.21.15.3:0/3216045439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958009570 con 0x7f79681575e0 2024-09-13T16:51:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f795fe00700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796813fdd0 0x7f796806a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f7966800700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f7968142b70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f796000be20 tx=0x7f7960009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f795e000700 1 -- 172.21.15.3:0/3216045439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000cc10 con 0x7f79681575e0 2024-09-13T16:51:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f795e000700 1 -- 172.21.15.3:0/3216045439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796000cd90 con 0x7f79681575e0 2024-09-13T16:51:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f795e000700 1 -- 172.21.15.3:0/3216045439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960014680 con 0x7f79681575e0 2024-09-13T16:51:03.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796806bd80 con 0x7f79681575e0 2024-09-13T16:51:03.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.573+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796806c580 con 0x7f79681575e0 2024-09-13T16:51:03.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.577+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796806c2a0 con 0x7f79681575e0 2024-09-13T16:51:03.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.577+0000 7f795e000700 1 -- 172.21.15.3:0/3216045439 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f796001d3b0 con 0x7f79681575e0 2024-09-13T16:51:03.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.581+0000 7f795e000700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7944066760 0x7f7944068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.581+0000 7f795e000700 1 -- 172.21.15.3:0/3216045439 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7960084360 con 0x7f79681575e0 2024-09-13T16:51:03.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.581+0000 7f7965e00700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7944066760 0x7f7944068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.581+0000 7f7965e00700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7944066760 0x7f7944068c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7958009150 tx=0x7f7958008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.585+0000 7f795e000700 1 -- 172.21.15.3:0/3216045439 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f796806c2a0 con 0x7f79681575e0 2024-09-13T16:51:03.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.765+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7968140240 con 0x7f7944066760 2024-09-13T16:51:03.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.781+0000 7f795e000700 1 -- 172.21.15.3:0/3216045439 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345056 (secure 0 0 0) 0x7f795800b960 con 0x7f7944066760 2024-09-13T16:51:03.792 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.789+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7944066760 msgr2=0x7f7944068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.789+0000 7f796d200700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7944066760 0x7f7944068c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7958009150 tx=0x7f7958008030 comp rx=0 tx=0).stop 2024-09-13T16:51:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.789+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 msgr2=0x7f7968142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.789+0000 7f796d200700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f7968142b70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f796000be20 tx=0x7f7960009ed0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.793+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 shutdown_connections 2024-09-13T16:51:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.793+0000 7f796d200700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7944066760 0x7f7944068c20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.793+0000 7f796d200700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79681575e0 0x7f7968142b70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.793+0000 7f796d200700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796813fdd0 0x7f796806a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.793+0000 7f796d200700 1 --2- 172.21.15.3:0/3216045439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79680cf460 0x7f79681f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.793+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 >> 172.21.15.3:0/3216045439 conn(0x7f796806d3c0 msgr2=0x7f796813f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.793+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 shutdown_connections 2024-09-13T16:51:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:03.793+0000 7f796d200700 1 -- 172.21.15.3:0/3216045439 wait complete. 2024-09-13T16:51:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:51:03.818 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-13T16:51:03.205099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227664,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754747056,"statfs":{"total":773094113280,"available":772860985344,"internally_reserved":0,"allocated":3383296,"data_stored":2044682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006306"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463496+0000","last_change":"2024-09-13T16:50:30.106124+0000","last_active":"2024-09-13T16:50:35.463496+0000","last_peered":"2024-09-13T16:50:35.463496+0000","last_clean":"2024-09-13T16:50:35.463496+0000","last_became_active":"2024-09-13T16:50:30.105874+0000","last_became_peered":"2024-09-13T16:50:30.105874+0000","last_unstale":"2024-09-13T16:50:35.463496+0000","last_undegraded":"2024-09-13T16:50:35.463496+0000","last_fullsized":"2024-09-13T16:50:35.463496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:38.882189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456272+0000","last_change":"2024-09-13T16:50:30.598817+0000","last_active":"2024-09-13T16:50:35.456272+0000","last_peered":"2024-09-13T16:50:35.456272+0000","last_clean":"2024-09-13T16:50:35.456272+0000","last_became_active":"2024-09-13T16:50:30.598483+0000","last_became_peered":"2024-09-13T16:50:30.598483+0000","last_unstale":"2024-09-13T16:50:35.456272+0000","last_undegraded":"2024-09-13T16:50:35.456272+0000","last_fullsized":"2024-09-13T16:50:35.456272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.511304+0000","last_change":"2024-09-13T16:50:35.507978+0000","last_active":"2024-09-13T16:50:35.511304+0000","last_peered":"2024-09-13T16:50:35.511304+0000","last_clean":"2024-09-13T16:50:35.511304+0000","last_became_active":"2024-09-13T16:50:35.506655+0000","last_became_peered":"2024-09-13T16:50:35.506655+0000","last_unstale":"2024-09-13T16:50:35.511304+0000","last_undegraded":"2024-09-13T16:50:35.511304+0000","last_fullsized":"2024-09-13T16:50:35.511304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:29.062934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456822+0000","last_change":"2024-09-13T16:50:30.101330+0000","last_active":"2024-09-13T16:50:35.456822+0000","last_peered":"2024-09-13T16:50:35.456822+0000","last_clean":"2024-09-13T16:50:35.456822+0000","last_became_active":"2024-09-13T16:50:30.101038+0000","last_became_peered":"2024-09-13T16:50:30.101038+0000","last_unstale":"2024-09-13T16:50:35.456822+0000","last_undegraded":"2024-09-13T16:50:35.456822+0000","last_fullsized":"2024-09-13T16:50:35.456822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463263+0000","last_change":"2024-09-13T16:50:30.609337+0000","last_active":"2024-09-13T16:50:35.463263+0000","last_peered":"2024-09-13T16:50:35.463263+0000","last_clean":"2024-09-13T16:50:35.463263+0000","last_became_active":"2024-09-13T16:50:30.609098+0000","last_became_peered":"2024-09-13T16:50:30.609098+0000","last_unstale":"2024-09-13T16:50:35.463263+0000","last_undegraded":"2024-09-13T16:50:35.463263+0000","last_fullsized":"2024-09-13T16:50:35.463263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498079+0000","last_change":"2024-09-13T16:50:34.633734+0000","last_active":"2024-09-13T16:50:35.498079+0000","last_peered":"2024-09-13T16:50:35.498079+0000","last_clean":"2024-09-13T16:50:35.498079+0000","last_became_active":"2024-09-13T16:50:34.632931+0000","last_became_peered":"2024-09-13T16:50:34.632931+0000","last_unstale":"2024-09-13T16:50:35.498079+0000","last_undegraded":"2024-09-13T16:50:35.498079+0000","last_fullsized":"2024-09-13T16:50:35.498079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:37.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500017+0000","last_change":"2024-09-13T16:50:30.108543+0000","last_active":"2024-09-13T16:50:35.500017+0000","last_peered":"2024-09-13T16:50:35.500017+0000","last_clean":"2024-09-13T16:50:35.500017+0000","last_became_active":"2024-09-13T16:50:30.108282+0000","last_became_peered":"2024-09-13T16:50:30.108282+0000","last_unstale":"2024-09-13T16:50:35.500017+0000","last_undegraded":"2024-09-13T16:50:35.500017+0000","last_fullsized":"2024-09-13T16:50:35.500017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092867+0000","last_change":"2024-09-13T16:50:30.594489+0000","last_active":"2024-09-13T16:50:35.092867+0000","last_peered":"2024-09-13T16:50:35.092867+0000","last_clean":"2024-09-13T16:50:35.092867+0000","last_became_active":"2024-09-13T16:50:30.593342+0000","last_became_peered":"2024-09-13T16:50:30.593342+0000","last_unstale":"2024-09-13T16:50:35.092867+0000","last_undegraded":"2024-09-13T16:50:35.092867+0000","last_fullsized":"2024-09-13T16:50:35.092867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500758+0000","last_change":"2024-09-13T16:50:34.633900+0000","last_active":"2024-09-13T16:50:35.500758+0000","last_peered":"2024-09-13T16:50:35.500758+0000","last_clean":"2024-09-13T16:50:35.500758+0000","last_became_active":"2024-09-13T16:50:34.633428+0000","last_became_peered":"2024-09-13T16:50:34.633428+0000","last_unstale":"2024-09-13T16:50:35.500758+0000","last_undegraded":"2024-09-13T16:50:35.500758+0000","last_fullsized":"2024-09-13T16:50:35.500758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:01.459147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499214+0000","last_change":"2024-09-13T16:50:30.119534+0000","last_active":"2024-09-13T16:50:35.499214+0000","last_peered":"2024-09-13T16:50:35.499214+0000","last_clean":"2024-09-13T16:50:35.499214+0000","last_became_active":"2024-09-13T16:50:30.119177+0000","last_became_peered":"2024-09-13T16:50:30.119177+0000","last_unstale":"2024-09-13T16:50:35.499214+0000","last_undegraded":"2024-09-13T16:50:35.499214+0000","last_fullsized":"2024-09-13T16:50:35.499214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092738+0000","last_change":"2024-09-13T16:50:30.604539+0000","last_active":"2024-09-13T16:50:35.092738+0000","last_peered":"2024-09-13T16:50:35.092738+0000","last_clean":"2024-09-13T16:50:35.092738+0000","last_became_active":"2024-09-13T16:50:30.604192+0000","last_became_peered":"2024-09-13T16:50:30.604192+0000","last_unstale":"2024-09-13T16:50:35.092738+0000","last_undegraded":"2024-09-13T16:50:35.092738+0000","last_fullsized":"2024-09-13T16:50:35.092738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500774+0000","last_change":"2024-09-13T16:50:34.632525+0000","last_active":"2024-09-13T16:50:35.500774+0000","last_peered":"2024-09-13T16:50:35.500774+0000","last_clean":"2024-09-13T16:50:35.500774+0000","last_became_active":"2024-09-13T16:50:34.632114+0000","last_became_peered":"2024-09-13T16:50:34.632114+0000","last_unstale":"2024-09-13T16:50:35.500774+0000","last_undegraded":"2024-09-13T16:50:35.500774+0000","last_fullsized":"2024-09-13T16:50:35.500774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091628+0000","last_change":"2024-09-13T16:50:30.611124+0000","last_active":"2024-09-13T16:50:35.091628+0000","last_peered":"2024-09-13T16:50:35.091628+0000","last_clean":"2024-09-13T16:50:35.091628+0000","last_became_active":"2024-09-13T16:50:30.607850+0000","last_became_peered":"2024-09-13T16:50:30.607850+0000","last_unstale":"2024-09-13T16:50:35.091628+0000","last_undegraded":"2024-09-13T16:50:35.091628+0000","last_fullsized":"2024-09-13T16:50:35.091628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499169+0000","last_change":"2024-09-13T16:50:30.106841+0000","last_active":"2024-09-13T16:50:35.499169+0000","last_peered":"2024-09-13T16:50:35.499169+0000","last_clean":"2024-09-13T16:50:35.499169+0000","last_became_active":"2024-09-13T16:50:30.106581+0000","last_became_peered":"2024-09-13T16:50:30.106581+0000","last_unstale":"2024-09-13T16:50:35.499169+0000","last_undegraded":"2024-09-13T16:50:35.499169+0000","last_fullsized":"2024-09-13T16:50:35.499169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501399+0000","last_change":"2024-09-13T16:50:34.633477+0000","last_active":"2024-09-13T16:50:35.501399+0000","last_peered":"2024-09-13T16:50:35.501399+0000","last_clean":"2024-09-13T16:50:35.501399+0000","last_became_active":"2024-09-13T16:50:34.633072+0000","last_became_peered":"2024-09-13T16:50:34.633072+0000","last_unstale":"2024-09-13T16:50:35.501399+0000","last_undegraded":"2024-09-13T16:50:35.501399+0000","last_fullsized":"2024-09-13T16:50:35.501399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:14.360209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456436+0000","last_change":"2024-09-13T16:50:30.611139+0000","last_active":"2024-09-13T16:50:35.456436+0000","last_peered":"2024-09-13T16:50:35.456436+0000","last_clean":"2024-09-13T16:50:35.456436+0000","last_became_active":"2024-09-13T16:50:30.610455+0000","last_became_peered":"2024-09-13T16:50:30.610455+0000","last_unstale":"2024-09-13T16:50:35.456436+0000","last_undegraded":"2024-09-13T16:50:35.456436+0000","last_fullsized":"2024-09-13T16:50:35.456436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402349+0000","last_change":"2024-09-13T16:50:30.113719+0000","last_active":"2024-09-13T16:50:35.402349+0000","last_peered":"2024-09-13T16:50:35.402349+0000","last_clean":"2024-09-13T16:50:35.402349+0000","last_became_active":"2024-09-13T16:50:30.111437+0000","last_became_peered":"2024-09-13T16:50:30.111437+0000","last_unstale":"2024-09-13T16:50:35.402349+0000","last_undegraded":"2024-09-13T16:50:35.402349+0000","last_fullsized":"2024-09-13T16:50:35.402349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501838+0000","last_change":"2024-09-13T16:50:34.638895+0000","last_active":"2024-09-13T16:50:35.501838+0000","last_peered":"2024-09-13T16:50:35.501838+0000","last_clean":"2024-09-13T16:50:35.501838+0000","last_became_active":"2024-09-13T16:50:34.638431+0000","last_became_peered":"2024-09-13T16:50:34.638431+0000","last_unstale":"2024-09-13T16:50:35.501838+0000","last_undegraded":"2024-09-13T16:50:35.501838+0000","last_fullsized":"2024-09-13T16:50:35.501838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:51.705201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.457086+0000","last_change":"2024-09-13T16:50:30.599950+0000","last_active":"2024-09-13T16:50:35.457086+0000","last_peered":"2024-09-13T16:50:35.457086+0000","last_clean":"2024-09-13T16:50:35.457086+0000","last_became_active":"2024-09-13T16:50:30.599401+0000","last_became_peered":"2024-09-13T16:50:30.599401+0000","last_unstale":"2024-09-13T16:50:35.457086+0000","last_undegraded":"2024-09-13T16:50:35.457086+0000","last_fullsized":"2024-09-13T16:50:35.457086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401297+0000","last_change":"2024-09-13T16:50:30.110698+0000","last_active":"2024-09-13T16:50:35.401297+0000","last_peered":"2024-09-13T16:50:35.401297+0000","last_clean":"2024-09-13T16:50:35.401297+0000","last_became_active":"2024-09-13T16:50:30.110343+0000","last_became_peered":"2024-09-13T16:50:30.110343+0000","last_unstale":"2024-09-13T16:50:35.401297+0000","last_undegraded":"2024-09-13T16:50:35.401297+0000","last_fullsized":"2024-09-13T16:50:35.401297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.512080+0000","last_change":"2024-09-13T16:50:35.510605+0000","last_active":"2024-09-13T16:50:35.512080+0000","last_peered":"2024-09-13T16:50:35.512080+0000","last_clean":"2024-09-13T16:50:35.512080+0000","last_became_active":"2024-09-13T16:50:35.510230+0000","last_became_peered":"2024-09-13T16:50:35.510230+0000","last_unstale":"2024-09-13T16:50:35.512080+0000","last_undegraded":"2024-09-13T16:50:35.512080+0000","last_fullsized":"2024-09-13T16:50:35.512080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500916+0000","last_change":"2024-09-13T16:50:30.608538+0000","last_active":"2024-09-13T16:50:35.500916+0000","last_peered":"2024-09-13T16:50:35.500916+0000","last_clean":"2024-09-13T16:50:35.500916+0000","last_became_active":"2024-09-13T16:50:30.608195+0000","last_became_peered":"2024-09-13T16:50:30.608195+0000","last_unstale":"2024-09-13T16:50:35.500916+0000","last_undegraded":"2024-09-13T16:50:35.500916+0000","last_fullsized":"2024-09-13T16:50:35.500916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:48.431406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.497470+0000","last_change":"2024-09-13T16:50:30.110135+0000","last_active":"2024-09-13T16:50:35.497470+0000","last_peered":"2024-09-13T16:50:35.497470+0000","last_clean":"2024-09-13T16:50:35.497470+0000","last_became_active":"2024-09-13T16:50:30.109520+0000","last_became_peered":"2024-09-13T16:50:30.109520+0000","last_unstale":"2024-09-13T16:50:35.497470+0000","last_undegraded":"2024-09-13T16:50:35.497470+0000","last_fullsized":"2024-09-13T16:50:35.497470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501389+0000","last_change":"2024-09-13T16:50:34.635782+0000","last_active":"2024-09-13T16:50:35.501389+0000","last_peered":"2024-09-13T16:50:35.501389+0000","last_clean":"2024-09-13T16:50:35.501389+0000","last_became_active":"2024-09-13T16:50:34.635349+0000","last_became_peered":"2024-09-13T16:50:34.635349+0000","last_unstale":"2024-09-13T16:50:35.501389+0000","last_undegraded":"2024-09-13T16:50:35.501389+0000","last_fullsized":"2024-09-13T16:50:35.501389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:58.754567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463122+0000","last_change":"2024-09-13T16:50:30.614804+0000","last_active":"2024-09-13T16:50:35.463122+0000","last_peered":"2024-09-13T16:50:35.463122+0000","last_clean":"2024-09-13T16:50:35.463122+0000","last_became_active":"2024-09-13T16:50:30.614458+0000","last_became_peered":"2024-09-13T16:50:30.614458+0000","last_unstale":"2024-09-13T16:50:35.463122+0000","last_undegraded":"2024-09-13T16:50:35.463122+0000","last_fullsized":"2024-09-13T16:50:35.463122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093301+0000","last_change":"2024-09-13T16:50:30.107253+0000","last_active":"2024-09-13T16:50:35.093301+0000","last_peered":"2024-09-13T16:50:35.093301+0000","last_clean":"2024-09-13T16:50:35.093301+0000","last_became_active":"2024-09-13T16:50:30.106448+0000","last_became_peered":"2024-09-13T16:50:30.106448+0000","last_unstale":"2024-09-13T16:50:35.093301+0000","last_undegraded":"2024-09-13T16:50:35.093301+0000","last_fullsized":"2024-09-13T16:50:35.093301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.507157+0000","last_change":"2024-09-13T16:50:35.504918+0000","last_active":"2024-09-13T16:50:35.507157+0000","last_peered":"2024-09-13T16:50:35.507157+0000","last_clean":"2024-09-13T16:50:35.507157+0000","last_became_active":"2024-09-13T16:50:35.504252+0000","last_became_peered":"2024-09-13T16:50:35.504252+0000","last_unstale":"2024-09-13T16:50:35.507157+0000","last_undegraded":"2024-09-13T16:50:35.507157+0000","last_fullsized":"2024-09-13T16:50:35.507157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:54.702371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456826+0000","last_change":"2024-09-13T16:50:30.610870+0000","last_active":"2024-09-13T16:50:35.456826+0000","last_peered":"2024-09-13T16:50:35.456826+0000","last_clean":"2024-09-13T16:50:35.456826+0000","last_became_active":"2024-09-13T16:50:30.610473+0000","last_became_peered":"2024-09-13T16:50:30.610473+0000","last_unstale":"2024-09-13T16:50:35.456826+0000","last_undegraded":"2024-09-13T16:50:35.456826+0000","last_fullsized":"2024-09-13T16:50:35.456826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.940724+0000","last_change":"2024-09-13T16:50:30.111250+0000","last_active":"2024-09-13T16:50:35.940724+0000","last_peered":"2024-09-13T16:50:35.940724+0000","last_clean":"2024-09-13T16:50:35.940724+0000","last_became_active":"2024-09-13T16:50:30.109959+0000","last_became_peered":"2024-09-13T16:50:30.109959+0000","last_unstale":"2024-09-13T16:50:35.940724+0000","last_undegraded":"2024-09-13T16:50:35.940724+0000","last_fullsized":"2024-09-13T16:50:35.940724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509887+0000","last_change":"2024-09-13T16:50:35.501491+0000","last_active":"2024-09-13T16:50:35.509887+0000","last_peered":"2024-09-13T16:50:35.509887+0000","last_clean":"2024-09-13T16:50:35.509887+0000","last_became_active":"2024-09-13T16:50:35.500108+0000","last_became_peered":"2024-09-13T16:50:35.500108+0000","last_unstale":"2024-09-13T16:50:35.509887+0000","last_undegraded":"2024-09-13T16:50:35.509887+0000","last_fullsized":"2024-09-13T16:50:35.509887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:30.666178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401804+0000","last_change":"2024-09-13T16:50:30.606888+0000","last_active":"2024-09-13T16:50:35.401804+0000","last_peered":"2024-09-13T16:50:35.401804+0000","last_clean":"2024-09-13T16:50:35.401804+0000","last_became_active":"2024-09-13T16:50:30.605360+0000","last_became_peered":"2024-09-13T16:50:30.605360+0000","last_unstale":"2024-09-13T16:50:35.401804+0000","last_undegraded":"2024-09-13T16:50:35.401804+0000","last_fullsized":"2024-09-13T16:50:35.401804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500383+0000","last_change":"2024-09-13T16:50:30.111119+0000","last_active":"2024-09-13T16:50:35.500383+0000","last_peered":"2024-09-13T16:50:35.500383+0000","last_clean":"2024-09-13T16:50:35.500383+0000","last_became_active":"2024-09-13T16:50:30.110892+0000","last_became_peered":"2024-09-13T16:50:30.110892+0000","last_unstale":"2024-09-13T16:50:35.500383+0000","last_undegraded":"2024-09-13T16:50:35.500383+0000","last_fullsized":"2024-09-13T16:50:35.500383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509685+0000","last_change":"2024-09-13T16:50:35.505149+0000","last_active":"2024-09-13T16:50:35.509685+0000","last_peered":"2024-09-13T16:50:35.509685+0000","last_clean":"2024-09-13T16:50:35.509685+0000","last_became_active":"2024-09-13T16:50:35.502086+0000","last_became_peered":"2024-09-13T16:50:35.502086+0000","last_unstale":"2024-09-13T16:50:35.509685+0000","last_undegraded":"2024-09-13T16:50:35.509685+0000","last_fullsized":"2024-09-13T16:50:35.509685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092835+0000","last_change":"2024-09-13T16:50:30.601700+0000","last_active":"2024-09-13T16:50:35.092835+0000","last_peered":"2024-09-13T16:50:35.092835+0000","last_clean":"2024-09-13T16:50:35.092835+0000","last_became_active":"2024-09-13T16:50:30.601432+0000","last_became_peered":"2024-09-13T16:50:30.601432+0000","last_unstale":"2024-09-13T16:50:35.092835+0000","last_undegraded":"2024-09-13T16:50:35.092835+0000","last_fullsized":"2024-09-13T16:50:35.092835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:25.758134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402007+0000","last_change":"2024-09-13T16:50:30.112636+0000","last_active":"2024-09-13T16:50:35.402007+0000","last_peered":"2024-09-13T16:50:35.402007+0000","last_clean":"2024-09-13T16:50:35.402007+0000","last_became_active":"2024-09-13T16:50:30.112121+0000","last_became_peered":"2024-09-13T16:50:30.112121+0000","last_unstale":"2024-09-13T16:50:35.402007+0000","last_undegraded":"2024-09-13T16:50:35.402007+0000","last_fullsized":"2024-09-13T16:50:35.402007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401192+0000","last_change":"2024-09-13T16:50:34.633860+0000","last_active":"2024-09-13T16:50:35.401192+0000","last_peered":"2024-09-13T16:50:35.401192+0000","last_clean":"2024-09-13T16:50:35.401192+0000","last_became_active":"2024-09-13T16:50:34.633435+0000","last_became_peered":"2024-09-13T16:50:34.633435+0000","last_unstale":"2024-09-13T16:50:35.401192+0000","last_undegraded":"2024-09-13T16:50:35.401192+0000","last_fullsized":"2024-09-13T16:50:35.401192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:27.883765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501232+0000","last_change":"2024-09-13T16:50:30.613267+0000","last_active":"2024-09-13T16:50:35.501232+0000","last_peered":"2024-09-13T16:50:35.501232+0000","last_clean":"2024-09-13T16:50:35.501232+0000","last_became_active":"2024-09-13T16:50:30.612861+0000","last_became_peered":"2024-09-13T16:50:30.612861+0000","last_unstale":"2024-09-13T16:50:35.501232+0000","last_undegraded":"2024-09-13T16:50:35.501232+0000","last_fullsized":"2024-09-13T16:50:35.501232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:27.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091946+0000","last_change":"2024-09-13T16:50:30.110348+0000","last_active":"2024-09-13T16:50:35.091946+0000","last_peered":"2024-09-13T16:50:35.091946+0000","last_clean":"2024-09-13T16:50:35.091946+0000","last_became_active":"2024-09-13T16:50:30.110041+0000","last_became_peered":"2024-09-13T16:50:30.110041+0000","last_unstale":"2024-09-13T16:50:35.091946+0000","last_undegraded":"2024-09-13T16:50:35.091946+0000","last_fullsized":"2024-09-13T16:50:35.091946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401674+0000","last_change":"2024-09-13T16:50:34.633493+0000","last_active":"2024-09-13T16:50:35.401674+0000","last_peered":"2024-09-13T16:50:35.401674+0000","last_clean":"2024-09-13T16:50:35.401674+0000","last_became_active":"2024-09-13T16:50:34.632442+0000","last_became_peered":"2024-09-13T16:50:34.632442+0000","last_unstale":"2024-09-13T16:50:35.401674+0000","last_undegraded":"2024-09-13T16:50:35.401674+0000","last_fullsized":"2024-09-13T16:50:35.401674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401509+0000","last_change":"2024-09-13T16:50:30.590118+0000","last_active":"2024-09-13T16:50:35.401509+0000","last_peered":"2024-09-13T16:50:35.401509+0000","last_clean":"2024-09-13T16:50:35.401509+0000","last_became_active":"2024-09-13T16:50:30.589880+0000","last_became_peered":"2024-09-13T16:50:30.589880+0000","last_unstale":"2024-09-13T16:50:35.401509+0000","last_undegraded":"2024-09-13T16:50:35.401509+0000","last_fullsized":"2024-09-13T16:50:35.401509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:32.908616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087743+0000","last_change":"2024-09-13T16:50:30.103427+0000","last_active":"2024-09-13T16:50:35.087743+0000","last_peered":"2024-09-13T16:50:35.087743+0000","last_clean":"2024-09-13T16:50:35.087743+0000","last_became_active":"2024-09-13T16:50:30.102964+0000","last_became_peered":"2024-09-13T16:50:30.102964+0000","last_unstale":"2024-09-13T16:50:35.087743+0000","last_undegraded":"2024-09-13T16:50:35.087743+0000","last_fullsized":"2024-09-13T16:50:35.087743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402109+0000","last_change":"2024-09-13T16:50:34.632823+0000","last_active":"2024-09-13T16:50:35.402109+0000","last_peered":"2024-09-13T16:50:35.402109+0000","last_clean":"2024-09-13T16:50:35.402109+0000","last_became_active":"2024-09-13T16:50:34.632261+0000","last_became_peered":"2024-09-13T16:50:34.632261+0000","last_unstale":"2024-09-13T16:50:35.402109+0000","last_undegraded":"2024-09-13T16:50:35.402109+0000","last_fullsized":"2024-09-13T16:50:35.402109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462419+0000","last_change":"2024-09-13T16:50:30.606638+0000","last_active":"2024-09-13T16:50:35.462419+0000","last_peered":"2024-09-13T16:50:35.462419+0000","last_clean":"2024-09-13T16:50:35.462419+0000","last_became_active":"2024-09-13T16:50:30.606375+0000","last_became_peered":"2024-09-13T16:50:30.606375+0000","last_unstale":"2024-09-13T16:50:35.462419+0000","last_undegraded":"2024-09-13T16:50:35.462419+0000","last_fullsized":"2024-09-13T16:50:35.462419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:27.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498015+0000","last_change":"2024-09-13T16:50:30.115629+0000","last_active":"2024-09-13T16:50:35.498015+0000","last_peered":"2024-09-13T16:50:35.498015+0000","last_clean":"2024-09-13T16:50:35.498015+0000","last_became_active":"2024-09-13T16:50:30.115336+0000","last_became_peered":"2024-09-13T16:50:30.115336+0000","last_unstale":"2024-09-13T16:50:35.498015+0000","last_undegraded":"2024-09-13T16:50:35.498015+0000","last_fullsized":"2024-09-13T16:50:35.498015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:47.419945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509728+0000","last_change":"2024-09-13T16:50:35.507866+0000","last_active":"2024-09-13T16:50:35.509728+0000","last_peered":"2024-09-13T16:50:35.509728+0000","last_clean":"2024-09-13T16:50:35.509728+0000","last_became_active":"2024-09-13T16:50:35.505548+0000","last_became_peered":"2024-09-13T16:50:35.505548+0000","last_unstale":"2024-09-13T16:50:35.509728+0000","last_undegraded":"2024-09-13T16:50:35.509728+0000","last_fullsized":"2024-09-13T16:50:35.509728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.455646+0000","last_change":"2024-09-13T16:50:30.596915+0000","last_active":"2024-09-13T16:50:35.455646+0000","last_peered":"2024-09-13T16:50:35.455646+0000","last_clean":"2024-09-13T16:50:35.455646+0000","last_became_active":"2024-09-13T16:50:30.596643+0000","last_became_peered":"2024-09-13T16:50:30.596643+0000","last_unstale":"2024-09-13T16:50:35.455646+0000","last_undegraded":"2024-09-13T16:50:35.455646+0000","last_fullsized":"2024-09-13T16:50:35.455646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092229+0000","last_change":"2024-09-13T16:50:30.115172+0000","last_active":"2024-09-13T16:50:35.092229+0000","last_peered":"2024-09-13T16:50:35.092229+0000","last_clean":"2024-09-13T16:50:35.092229+0000","last_became_active":"2024-09-13T16:50:30.114633+0000","last_became_peered":"2024-09-13T16:50:30.114633+0000","last_unstale":"2024-09-13T16:50:35.092229+0000","last_undegraded":"2024-09-13T16:50:35.092229+0000","last_fullsized":"2024-09-13T16:50:35.092229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498080+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:50:35.498080+0000","last_peered":"2024-09-13T16:50:35.498080+0000","last_clean":"2024-09-13T16:50:35.498080+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:50:35.498080+0000","last_undegraded":"2024-09-13T16:50:35.498080+0000","last_fullsized":"2024-09-13T16:50:35.498080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.506121+0000","last_change":"2024-09-13T16:50:35.502219+0000","last_active":"2024-09-13T16:50:35.506121+0000","last_peered":"2024-09-13T16:50:35.506121+0000","last_clean":"2024-09-13T16:50:35.506121+0000","last_became_active":"2024-09-13T16:50:35.501877+0000","last_became_peered":"2024-09-13T16:50:35.501877+0000","last_unstale":"2024-09-13T16:50:35.506121+0000","last_undegraded":"2024-09-13T16:50:35.506121+0000","last_fullsized":"2024-09-13T16:50:35.506121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:37.440533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500614+0000","last_change":"2024-09-13T16:50:30.604143+0000","last_active":"2024-09-13T16:50:35.500614+0000","last_peered":"2024-09-13T16:50:35.500614+0000","last_clean":"2024-09-13T16:50:35.500614+0000","last_became_active":"2024-09-13T16:50:30.603882+0000","last_became_peered":"2024-09-13T16:50:30.603882+0000","last_unstale":"2024-09-13T16:50:35.500614+0000","last_undegraded":"2024-09-13T16:50:35.500614+0000","last_fullsized":"2024-09-13T16:50:35.500614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:31.996725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087857+0000","last_change":"2024-09-13T16:50:30.116691+0000","last_active":"2024-09-13T16:50:35.087857+0000","last_peered":"2024-09-13T16:50:35.087857+0000","last_clean":"2024-09-13T16:50:35.087857+0000","last_became_active":"2024-09-13T16:50:30.115262+0000","last_became_peered":"2024-09-13T16:50:30.115262+0000","last_unstale":"2024-09-13T16:50:35.087857+0000","last_undegraded":"2024-09-13T16:50:35.087857+0000","last_fullsized":"2024-09-13T16:50:35.087857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.506161+0000","last_change":"2024-09-13T16:50:35.499612+0000","last_active":"2024-09-13T16:50:35.506161+0000","last_peered":"2024-09-13T16:50:35.506161+0000","last_clean":"2024-09-13T16:50:35.506161+0000","last_became_active":"2024-09-13T16:50:35.498882+0000","last_became_peered":"2024-09-13T16:50:35.498882+0000","last_unstale":"2024-09-13T16:50:35.506161+0000","last_undegraded":"2024-09-13T16:50:35.506161+0000","last_fullsized":"2024-09-13T16:50:35.506161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:14.360209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092696+0000","last_change":"2024-09-13T16:50:30.601337+0000","last_active":"2024-09-13T16:50:35.092696+0000","last_peered":"2024-09-13T16:50:35.092696+0000","last_clean":"2024-09-13T16:50:35.092696+0000","last_became_active":"2024-09-13T16:50:30.601073+0000","last_became_peered":"2024-09-13T16:50:30.601073+0000","last_unstale":"2024-09-13T16:50:35.092696+0000","last_undegraded":"2024-09-13T16:50:35.092696+0000","last_fullsized":"2024-09-13T16:50:35.092696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.117003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402285+0000","last_change":"2024-09-13T16:50:30.109801+0000","last_active":"2024-09-13T16:50:35.402285+0000","last_peered":"2024-09-13T16:50:35.402285+0000","last_clean":"2024-09-13T16:50:35.402285+0000","last_became_active":"2024-09-13T16:50:30.107936+0000","last_became_peered":"2024-09-13T16:50:30.107936+0000","last_unstale":"2024-09-13T16:50:35.402285+0000","last_undegraded":"2024-09-13T16:50:35.402285+0000","last_fullsized":"2024-09-13T16:50:35.402285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462892+0000","last_change":"2024-09-13T16:50:34.640936+0000","last_active":"2024-09-13T16:50:35.462892+0000","last_peered":"2024-09-13T16:50:35.462892+0000","last_clean":"2024-09-13T16:50:35.462892+0000","last_became_active":"2024-09-13T16:50:34.640606+0000","last_became_peered":"2024-09-13T16:50:34.640606+0000","last_unstale":"2024-09-13T16:50:35.462892+0000","last_undegraded":"2024-09-13T16:50:35.462892+0000","last_fullsized":"2024-09-13T16:50:35.462892+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500787+0000","last_change":"2024-09-13T16:50:30.620140+0000","last_active":"2024-09-13T16:50:35.500787+0000","last_peered":"2024-09-13T16:50:35.500787+0000","last_clean":"2024-09-13T16:50:35.500787+0000","last_became_active":"2024-09-13T16:50:30.619960+0000","last_became_peered":"2024-09-13T16:50:30.619960+0000","last_unstale":"2024-09-13T16:50:35.500787+0000","last_undegraded":"2024-09-13T16:50:35.500787+0000","last_fullsized":"2024-09-13T16:50:35.500787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:28.715573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.088162+0000","last_change":"2024-09-13T16:50:30.104959+0000","last_active":"2024-09-13T16:50:35.088162+0000","last_peered":"2024-09-13T16:50:35.088162+0000","last_clean":"2024-09-13T16:50:35.088162+0000","last_became_active":"2024-09-13T16:50:30.104709+0000","last_became_peered":"2024-09-13T16:50:30.104709+0000","last_unstale":"2024-09-13T16:50:35.088162+0000","last_undegraded":"2024-09-13T16:50:35.088162+0000","last_fullsized":"2024-09-13T16:50:35.088162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.828286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.505477+0000","last_change":"2024-09-13T16:50:35.501120+0000","last_active":"2024-09-13T16:50:35.505477+0000","last_peered":"2024-09-13T16:50:35.505477+0000","last_clean":"2024-09-13T16:50:35.505477+0000","last_became_active":"2024-09-13T16:50:35.500500+0000","last_became_peered":"2024-09-13T16:50:35.500500+0000","last_unstale":"2024-09-13T16:50:35.505477+0000","last_undegraded":"2024-09-13T16:50:35.505477+0000","last_fullsized":"2024-09-13T16:50:35.505477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499025+0000","last_change":"2024-09-13T16:50:30.615966+0000","last_active":"2024-09-13T16:50:35.499025+0000","last_peered":"2024-09-13T16:50:35.499025+0000","last_clean":"2024-09-13T16:50:35.499025+0000","last_became_active":"2024-09-13T16:50:30.615750+0000","last_became_peered":"2024-09-13T16:50:30.615750+0000","last_unstale":"2024-09-13T16:50:35.499025+0000","last_undegraded":"2024-09-13T16:50:35.499025+0000","last_fullsized":"2024-09-13T16:50:35.499025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501690+0000","last_change":"2024-09-13T16:50:30.112955+0000","last_active":"2024-09-13T16:50:35.501690+0000","last_peered":"2024-09-13T16:50:35.501690+0000","last_clean":"2024-09-13T16:50:35.501690+0000","last_became_active":"2024-09-13T16:50:30.111633+0000","last_became_peered":"2024-09-13T16:50:30.111633+0000","last_unstale":"2024-09-13T16:50:35.501690+0000","last_undegraded":"2024-09-13T16:50:35.501690+0000","last_fullsized":"2024-09-13T16:50:35.501690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499166+0000","last_change":"2024-09-13T16:50:34.636875+0000","last_active":"2024-09-13T16:50:35.499166+0000","last_peered":"2024-09-13T16:50:35.499166+0000","last_clean":"2024-09-13T16:50:35.499166+0000","last_became_active":"2024-09-13T16:50:34.636350+0000","last_became_peered":"2024-09-13T16:50:34.636350+0000","last_unstale":"2024-09-13T16:50:35.499166+0000","last_undegraded":"2024-09-13T16:50:35.499166+0000","last_fullsized":"2024-09-13T16:50:35.499166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:32.980957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498929+0000","last_change":"2024-09-13T16:50:30.617707+0000","last_active":"2024-09-13T16:50:35.498929+0000","last_peered":"2024-09-13T16:50:35.498929+0000","last_clean":"2024-09-13T16:50:35.498929+0000","last_became_active":"2024-09-13T16:50:30.617279+0000","last_became_peered":"2024-09-13T16:50:30.617279+0000","last_unstale":"2024-09-13T16:50:35.498929+0000","last_undegraded":"2024-09-13T16:50:35.498929+0000","last_fullsized":"2024-09-13T16:50:35.498929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087386+0000","last_change":"2024-09-13T16:50:30.107834+0000","last_active":"2024-09-13T16:50:35.087386+0000","last_peered":"2024-09-13T16:50:35.087386+0000","last_clean":"2024-09-13T16:50:35.087386+0000","last_became_active":"2024-09-13T16:50:30.107457+0000","last_became_peered":"2024-09-13T16:50:30.107457+0000","last_unstale":"2024-09-13T16:50:35.087386+0000","last_undegraded":"2024-09-13T16:50:35.087386+0000","last_fullsized":"2024-09-13T16:50:35.087386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463011+0000","last_change":"2024-09-13T16:50:34.635554+0000","last_active":"2024-09-13T16:50:35.463011+0000","last_peered":"2024-09-13T16:50:35.463011+0000","last_clean":"2024-09-13T16:50:35.463011+0000","last_became_active":"2024-09-13T16:50:34.635194+0000","last_became_peered":"2024-09-13T16:50:34.635194+0000","last_unstale":"2024-09-13T16:50:35.463011+0000","last_undegraded":"2024-09-13T16:50:35.463011+0000","last_fullsized":"2024-09-13T16:50:35.463011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:54.702371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091123+0000","last_change":"2024-09-13T16:50:30.612793+0000","last_active":"2024-09-13T16:50:35.091123+0000","last_peered":"2024-09-13T16:50:35.091123+0000","last_clean":"2024-09-13T16:50:35.091123+0000","last_became_active":"2024-09-13T16:50:30.612243+0000","last_became_peered":"2024-09-13T16:50:30.612243+0000","last_unstale":"2024-09-13T16:50:35.091123+0000","last_undegraded":"2024-09-13T16:50:35.091123+0000","last_fullsized":"2024-09-13T16:50:35.091123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091094+0000","last_change":"2024-09-13T16:50:30.097963+0000","last_active":"2024-09-13T16:50:35.091094+0000","last_peered":"2024-09-13T16:50:35.091094+0000","last_clean":"2024-09-13T16:50:35.091094+0000","last_became_active":"2024-09-13T16:50:30.097015+0000","last_became_peered":"2024-09-13T16:50:30.097015+0000","last_unstale":"2024-09-13T16:50:35.091094+0000","last_undegraded":"2024-09-13T16:50:35.091094+0000","last_fullsized":"2024-09-13T16:50:35.091094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.514416+0000","last_change":"2024-09-13T16:50:35.512427+0000","last_active":"2024-09-13T16:50:35.514416+0000","last_peered":"2024-09-13T16:50:35.514416+0000","last_clean":"2024-09-13T16:50:35.514416+0000","last_became_active":"2024-09-13T16:50:35.512074+0000","last_became_peered":"2024-09-13T16:50:35.512074+0000","last_unstale":"2024-09-13T16:50:35.514416+0000","last_undegraded":"2024-09-13T16:50:35.514416+0000","last_fullsized":"2024-09-13T16:50:35.514416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:37.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498993+0000","last_change":"2024-09-13T16:50:30.617270+0000","last_active":"2024-09-13T16:50:35.498993+0000","last_peered":"2024-09-13T16:50:35.498993+0000","last_clean":"2024-09-13T16:50:35.498993+0000","last_became_active":"2024-09-13T16:50:30.617047+0000","last_became_peered":"2024-09-13T16:50:30.617047+0000","last_unstale":"2024-09-13T16:50:35.498993+0000","last_undegraded":"2024-09-13T16:50:35.498993+0000","last_fullsized":"2024-09-13T16:50:35.498993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091892+0000","last_change":"2024-09-13T16:50:30.109703+0000","last_active":"2024-09-13T16:50:35.091892+0000","last_peered":"2024-09-13T16:50:35.091892+0000","last_clean":"2024-09-13T16:50:35.091892+0000","last_became_active":"2024-09-13T16:50:30.109427+0000","last_became_peered":"2024-09-13T16:50:30.109427+0000","last_unstale":"2024-09-13T16:50:35.091892+0000","last_undegraded":"2024-09-13T16:50:35.091892+0000","last_fullsized":"2024-09-13T16:50:35.091892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509958+0000","last_change":"2024-09-13T16:50:35.501616+0000","last_active":"2024-09-13T16:50:35.509958+0000","last_peered":"2024-09-13T16:50:35.509958+0000","last_clean":"2024-09-13T16:50:35.509958+0000","last_became_active":"2024-09-13T16:50:35.500854+0000","last_became_peered":"2024-09-13T16:50:35.500854+0000","last_unstale":"2024-09-13T16:50:35.509958+0000","last_undegraded":"2024-09-13T16:50:35.509958+0000","last_fullsized":"2024-09-13T16:50:35.509958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:18.549426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462681+0000","last_change":"2024-09-13T16:50:30.619507+0000","last_active":"2024-09-13T16:50:35.462681+0000","last_peered":"2024-09-13T16:50:35.462681+0000","last_clean":"2024-09-13T16:50:35.462681+0000","last_became_active":"2024-09-13T16:50:30.619122+0000","last_became_peered":"2024-09-13T16:50:30.619122+0000","last_unstale":"2024-09-13T16:50:35.462681+0000","last_undegraded":"2024-09-13T16:50:35.462681+0000","last_fullsized":"2024-09-13T16:50:35.462681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:00.203031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463139+0000","last_change":"2024-09-13T16:50:30.099206+0000","last_active":"2024-09-13T16:50:35.463139+0000","last_peered":"2024-09-13T16:50:35.463139+0000","last_clean":"2024-09-13T16:50:35.463139+0000","last_became_active":"2024-09-13T16:50:30.098688+0000","last_became_peered":"2024-09-13T16:50:30.098688+0000","last_unstale":"2024-09-13T16:50:35.463139+0000","last_undegraded":"2024-09-13T16:50:35.463139+0000","last_fullsized":"2024-09-13T16:50:35.463139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.505292+0000","last_change":"2024-09-13T16:50:35.502406+0000","last_active":"2024-09-13T16:50:35.505292+0000","last_peered":"2024-09-13T16:50:35.505292+0000","last_clean":"2024-09-13T16:50:35.505292+0000","last_became_active":"2024-09-13T16:50:35.502014+0000","last_became_peered":"2024-09-13T16:50:35.502014+0000","last_unstale":"2024-09-13T16:50:35.505292+0000","last_undegraded":"2024-09-13T16:50:35.505292+0000","last_fullsized":"2024-09-13T16:50:35.505292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498570+0000","last_change":"2024-09-13T16:50:30.613053+0000","last_active":"2024-09-13T16:50:35.498570+0000","last_peered":"2024-09-13T16:50:35.498570+0000","last_clean":"2024-09-13T16:50:35.498570+0000","last_became_active":"2024-09-13T16:50:30.612775+0000","last_became_peered":"2024-09-13T16:50:30.612775+0000","last_unstale":"2024-09-13T16:50:35.498570+0000","last_undegraded":"2024-09-13T16:50:35.498570+0000","last_fullsized":"2024-09-13T16:50:35.498570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:54.244738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T16:50:47.108380+0000","last_change":"2024-09-13T16:50:30.111095+0000","last_active":"2024-09-13T16:50:47.108380+0000","last_peered":"2024-09-13T16:50:47.108380+0000","last_clean":"2024-09-13T16:50:47.108380+0000","last_became_active":"2024-09-13T16:50:30.110886+0000","last_became_peered":"2024-09-13T16:50:30.110886+0000","last_unstale":"2024-09-13T16:50:47.108380+0000","last_undegraded":"2024-09-13T16:50:47.108380+0000","last_fullsized":"2024-09-13T16:50:47.108380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401424+0000","last_change":"2024-09-13T16:50:34.641283+0000","last_active":"2024-09-13T16:50:35.401424+0000","last_peered":"2024-09-13T16:50:35.401424+0000","last_clean":"2024-09-13T16:50:35.401424+0000","last_became_active":"2024-09-13T16:50:34.636369+0000","last_became_peered":"2024-09-13T16:50:34.636369+0000","last_unstale":"2024-09-13T16:50:35.401424+0000","last_undegraded":"2024-09-13T16:50:35.401424+0000","last_fullsized":"2024-09-13T16:50:35.401424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:46.586048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.502226+0000","last_change":"2024-09-13T16:50:30.611541+0000","last_active":"2024-09-13T16:50:35.502226+0000","last_peered":"2024-09-13T16:50:35.502226+0000","last_clean":"2024-09-13T16:50:35.502226+0000","last_became_active":"2024-09-13T16:50:30.611046+0000","last_became_peered":"2024-09-13T16:50:30.611046+0000","last_unstale":"2024-09-13T16:50:35.502226+0000","last_undegraded":"2024-09-13T16:50:35.502226+0000","last_fullsized":"2024-09-13T16:50:35.502226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:51.423821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402184+0000","last_change":"2024-09-13T16:50:30.111117+0000","last_active":"2024-09-13T16:50:35.402184+0000","last_peered":"2024-09-13T16:50:35.402184+0000","last_clean":"2024-09-13T16:50:35.402184+0000","last_became_active":"2024-09-13T16:50:30.110857+0000","last_became_peered":"2024-09-13T16:50:30.110857+0000","last_unstale":"2024-09-13T16:50:35.402184+0000","last_undegraded":"2024-09-13T16:50:35.402184+0000","last_fullsized":"2024-09-13T16:50:35.402184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500960+0000","last_change":"2024-09-13T16:50:34.634925+0000","last_active":"2024-09-13T16:50:35.500960+0000","last_peered":"2024-09-13T16:50:35.500960+0000","last_clean":"2024-09-13T16:50:35.500960+0000","last_became_active":"2024-09-13T16:50:34.634329+0000","last_became_peered":"2024-09-13T16:50:34.634329+0000","last_unstale":"2024-09-13T16:50:35.500960+0000","last_undegraded":"2024-09-13T16:50:35.500960+0000","last_fullsized":"2024-09-13T16:50:35.500960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:30.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463375+0000","last_change":"2024-09-13T16:50:30.609227+0000","last_active":"2024-09-13T16:50:35.463375+0000","last_peered":"2024-09-13T16:50:35.463375+0000","last_clean":"2024-09-13T16:50:35.463375+0000","last_became_active":"2024-09-13T16:50:30.608863+0000","last_became_peered":"2024-09-13T16:50:30.608863+0000","last_unstale":"2024-09-13T16:50:35.463375+0000","last_undegraded":"2024-09-13T16:50:35.463375+0000","last_fullsized":"2024-09-13T16:50:35.463375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093039+0000","last_change":"2024-09-13T16:50:30.109198+0000","last_active":"2024-09-13T16:50:35.093039+0000","last_peered":"2024-09-13T16:50:35.093039+0000","last_clean":"2024-09-13T16:50:35.093039+0000","last_became_active":"2024-09-13T16:50:30.108896+0000","last_became_peered":"2024-09-13T16:50:30.108896+0000","last_unstale":"2024-09-13T16:50:35.093039+0000","last_undegraded":"2024-09-13T16:50:35.093039+0000","last_fullsized":"2024-09-13T16:50:35.093039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456883+0000","last_change":"2024-09-13T16:50:34.630350+0000","last_active":"2024-09-13T16:50:35.456883+0000","last_peered":"2024-09-13T16:50:35.456883+0000","last_clean":"2024-09-13T16:50:35.456883+0000","last_became_active":"2024-09-13T16:50:34.626215+0000","last_became_peered":"2024-09-13T16:50:34.626215+0000","last_unstale":"2024-09-13T16:50:35.456883+0000","last_undegraded":"2024-09-13T16:50:35.456883+0000","last_fullsized":"2024-09-13T16:50:35.456883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:51.705201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092550+0000","last_change":"2024-09-13T16:50:30.603528+0000","last_active":"2024-09-13T16:50:35.092550+0000","last_peered":"2024-09-13T16:50:35.092550+0000","last_clean":"2024-09-13T16:50:35.092550+0000","last_became_active":"2024-09-13T16:50:30.603265+0000","last_became_peered":"2024-09-13T16:50:30.603265+0000","last_unstale":"2024-09-13T16:50:35.092550+0000","last_undegraded":"2024-09-13T16:50:35.092550+0000","last_fullsized":"2024-09-13T16:50:35.092550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093205+0000","last_change":"2024-09-13T16:50:30.112126+0000","last_active":"2024-09-13T16:50:35.093205+0000","last_peered":"2024-09-13T16:50:35.093205+0000","last_clean":"2024-09-13T16:50:35.093205+0000","last_became_active":"2024-09-13T16:50:30.111844+0000","last_became_peered":"2024-09-13T16:50:30.111844+0000","last_unstale":"2024-09-13T16:50:35.093205+0000","last_undegraded":"2024-09-13T16:50:35.093205+0000","last_fullsized":"2024-09-13T16:50:35.093205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.511284+0000","last_change":"2024-09-13T16:50:35.507739+0000","last_active":"2024-09-13T16:50:35.511284+0000","last_peered":"2024-09-13T16:50:35.511284+0000","last_clean":"2024-09-13T16:50:35.511284+0000","last_became_active":"2024-09-13T16:50:35.505030+0000","last_became_peered":"2024-09-13T16:50:35.505030+0000","last_unstale":"2024-09-13T16:50:35.511284+0000","last_undegraded":"2024-09-13T16:50:35.511284+0000","last_fullsized":"2024-09-13T16:50:35.511284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:29.316344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509712+0000","last_change":"2024-09-13T16:50:35.506793+0000","last_active":"2024-09-13T16:50:35.509712+0000","last_peered":"2024-09-13T16:50:35.509712+0000","last_clean":"2024-09-13T16:50:35.509712+0000","last_became_active":"2024-09-13T16:50:35.505380+0000","last_became_peered":"2024-09-13T16:50:35.505380+0000","last_unstale":"2024-09-13T16:50:35.509712+0000","last_undegraded":"2024-09-13T16:50:35.509712+0000","last_fullsized":"2024-09-13T16:50:35.509712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:58.754567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.400877+0000","last_change":"2024-09-13T16:50:30.601660+0000","last_active":"2024-09-13T16:50:35.400877+0000","last_peered":"2024-09-13T16:50:35.400877+0000","last_clean":"2024-09-13T16:50:35.400877+0000","last_became_active":"2024-09-13T16:50:30.601315+0000","last_became_peered":"2024-09-13T16:50:30.601315+0000","last_unstale":"2024-09-13T16:50:35.400877+0000","last_undegraded":"2024-09-13T16:50:35.400877+0000","last_fullsized":"2024-09-13T16:50:35.400877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091503+0000","last_change":"2024-09-13T16:50:30.098656+0000","last_active":"2024-09-13T16:50:35.091503+0000","last_peered":"2024-09-13T16:50:35.091503+0000","last_clean":"2024-09-13T16:50:35.091503+0000","last_became_active":"2024-09-13T16:50:30.098137+0000","last_became_peered":"2024-09-13T16:50:30.098137+0000","last_unstale":"2024-09-13T16:50:35.091503+0000","last_undegraded":"2024-09-13T16:50:35.091503+0000","last_fullsized":"2024-09-13T16:50:35.091503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093095+0000","last_change":"2024-09-13T16:50:34.632360+0000","last_active":"2024-09-13T16:50:35.093095+0000","last_peered":"2024-09-13T16:50:35.093095+0000","last_clean":"2024-09-13T16:50:35.093095+0000","last_became_active":"2024-09-13T16:50:34.631958+0000","last_became_peered":"2024-09-13T16:50:34.631958+0000","last_unstale":"2024-09-13T16:50:35.093095+0000","last_undegraded":"2024-09-13T16:50:35.093095+0000","last_fullsized":"2024-09-13T16:50:35.093095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:30.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456997+0000","last_change":"2024-09-13T16:50:30.609936+0000","last_active":"2024-09-13T16:50:35.456997+0000","last_peered":"2024-09-13T16:50:35.456997+0000","last_clean":"2024-09-13T16:50:35.456997+0000","last_became_active":"2024-09-13T16:50:30.609683+0000","last_became_peered":"2024-09-13T16:50:30.609683+0000","last_unstale":"2024-09-13T16:50:35.456997+0000","last_undegraded":"2024-09-13T16:50:35.456997+0000","last_fullsized":"2024-09-13T16:50:35.456997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:32.908616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456931+0000","last_change":"2024-09-13T16:50:30.103999+0000","last_active":"2024-09-13T16:50:35.456931+0000","last_peered":"2024-09-13T16:50:35.456931+0000","last_clean":"2024-09-13T16:50:35.456931+0000","last_became_active":"2024-09-13T16:50:30.103531+0000","last_became_peered":"2024-09-13T16:50:30.103531+0000","last_unstale":"2024-09-13T16:50:35.456931+0000","last_undegraded":"2024-09-13T16:50:35.456931+0000","last_fullsized":"2024-09-13T16:50:35.456931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.828286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091972+0000","last_change":"2024-09-13T16:50:30.103767+0000","last_active":"2024-09-13T16:50:35.091972+0000","last_peered":"2024-09-13T16:50:35.091972+0000","last_clean":"2024-09-13T16:50:35.091972+0000","last_became_active":"2024-09-13T16:50:30.103519+0000","last_became_peered":"2024-09-13T16:50:30.103519+0000","last_unstale":"2024-09-13T16:50:35.091972+0000","last_undegraded":"2024-09-13T16:50:35.091972+0000","last_fullsized":"2024-09-13T16:50:35.091972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463060+0000","last_change":"2024-09-13T16:50:30.618557+0000","last_active":"2024-09-13T16:50:35.463060+0000","last_peered":"2024-09-13T16:50:35.463060+0000","last_clean":"2024-09-13T16:50:35.463060+0000","last_became_active":"2024-09-13T16:50:30.618308+0000","last_became_peered":"2024-09-13T16:50:30.618308+0000","last_unstale":"2024-09-13T16:50:35.463060+0000","last_undegraded":"2024-09-13T16:50:35.463060+0000","last_fullsized":"2024-09-13T16:50:35.463060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:45.160388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087606+0000","last_change":"2024-09-13T16:50:34.639885+0000","last_active":"2024-09-13T16:50:35.087606+0000","last_peered":"2024-09-13T16:50:35.087606+0000","last_clean":"2024-09-13T16:50:35.087606+0000","last_became_active":"2024-09-13T16:50:34.639399+0000","last_became_peered":"2024-09-13T16:50:34.639399+0000","last_unstale":"2024-09-13T16:50:35.087606+0000","last_undegraded":"2024-09-13T16:50:35.087606+0000","last_fullsized":"2024-09-13T16:50:35.087606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:36.425638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456657+0000","last_change":"2024-09-13T16:50:30.102739+0000","last_active":"2024-09-13T16:50:35.456657+0000","last_peered":"2024-09-13T16:50:35.456657+0000","last_clean":"2024-09-13T16:50:35.456657+0000","last_became_active":"2024-09-13T16:50:30.102448+0000","last_became_peered":"2024-09-13T16:50:30.102448+0000","last_unstale":"2024-09-13T16:50:35.456657+0000","last_undegraded":"2024-09-13T16:50:35.456657+0000","last_fullsized":"2024-09-13T16:50:35.456657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:38.882189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501017+0000","last_change":"2024-09-13T16:50:30.597447+0000","last_active":"2024-09-13T16:50:35.501017+0000","last_peered":"2024-09-13T16:50:35.501017+0000","last_clean":"2024-09-13T16:50:35.501017+0000","last_became_active":"2024-09-13T16:50:30.592891+0000","last_became_peered":"2024-09-13T16:50:30.592891+0000","last_unstale":"2024-09-13T16:50:35.501017+0000","last_undegraded":"2024-09-13T16:50:35.501017+0000","last_fullsized":"2024-09-13T16:50:35.501017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:13.892340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456015+0000","last_change":"2024-09-13T16:50:34.630131+0000","last_active":"2024-09-13T16:50:35.456015+0000","last_peered":"2024-09-13T16:50:35.456015+0000","last_clean":"2024-09-13T16:50:35.456015+0000","last_became_active":"2024-09-13T16:50:34.629601+0000","last_became_peered":"2024-09-13T16:50:34.629601+0000","last_unstale":"2024-09-13T16:50:35.456015+0000","last_undegraded":"2024-09-13T16:50:35.456015+0000","last_fullsized":"2024-09-13T16:50:35.456015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:38.058431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.090962+0000","last_change":"2024-09-13T16:50:30.612982+0000","last_active":"2024-09-13T16:50:35.090962+0000","last_peered":"2024-09-13T16:50:35.090962+0000","last_clean":"2024-09-13T16:50:35.090962+0000","last_became_active":"2024-09-13T16:50:30.612577+0000","last_became_peered":"2024-09-13T16:50:30.612577+0000","last_unstale":"2024-09-13T16:50:35.090962+0000","last_undegraded":"2024-09-13T16:50:35.090962+0000","last_fullsized":"2024-09-13T16:50:35.090962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087762+0000","last_change":"2024-09-13T16:50:30.594530+0000","last_active":"2024-09-13T16:50:35.087762+0000","last_peered":"2024-09-13T16:50:35.087762+0000","last_clean":"2024-09-13T16:50:35.087762+0000","last_became_active":"2024-09-13T16:50:30.594248+0000","last_became_peered":"2024-09-13T16:50:30.594248+0000","last_unstale":"2024-09-13T16:50:35.087762+0000","last_undegraded":"2024-09-13T16:50:35.087762+0000","last_fullsized":"2024-09-13T16:50:35.087762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:22.950313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091814+0000","last_change":"2024-09-13T16:50:30.602412+0000","last_active":"2024-09-13T16:50:35.091814+0000","last_peered":"2024-09-13T16:50:35.091814+0000","last_clean":"2024-09-13T16:50:35.091814+0000","last_became_active":"2024-09-13T16:50:30.602146+0000","last_became_peered":"2024-09-13T16:50:30.602146+0000","last_unstale":"2024-09-13T16:50:35.091814+0000","last_undegraded":"2024-09-13T16:50:35.091814+0000","last_fullsized":"2024-09-13T16:50:35.091814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:35.586398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462908+0000","last_change":"2024-09-13T16:50:30.606003+0000","last_active":"2024-09-13T16:50:35.462908+0000","last_peered":"2024-09-13T16:50:35.462908+0000","last_clean":"2024-09-13T16:50:35.462908+0000","last_became_active":"2024-09-13T16:50:30.605562+0000","last_became_peered":"2024-09-13T16:50:30.605562+0000","last_unstale":"2024-09-13T16:50:35.462908+0000","last_undegraded":"2024-09-13T16:50:35.462908+0000","last_fullsized":"2024-09-13T16:50:35.462908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456431+0000","last_change":"2024-09-13T16:50:30.617701+0000","last_active":"2024-09-13T16:50:35.456431+0000","last_peered":"2024-09-13T16:50:35.456431+0000","last_clean":"2024-09-13T16:50:35.456431+0000","last_became_active":"2024-09-13T16:50:30.616581+0000","last_became_peered":"2024-09-13T16:50:30.616581+0000","last_unstale":"2024-09-13T16:50:35.456431+0000","last_undegraded":"2024-09-13T16:50:35.456431+0000","last_fullsized":"2024-09-13T16:50:35.456431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:52.041529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401854+0000","last_change":"2024-09-13T16:50:30.599260+0000","last_active":"2024-09-13T16:50:35.401854+0000","last_peered":"2024-09-13T16:50:35.401854+0000","last_clean":"2024-09-13T16:50:35.401854+0000","last_became_active":"2024-09-13T16:50:30.599018+0000","last_became_peered":"2024-09-13T16:50:30.599018+0000","last_unstale":"2024-09-13T16:50:35.401854+0000","last_undegraded":"2024-09-13T16:50:35.401854+0000","last_fullsized":"2024-09-13T16:50:35.401854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093088+0000","last_change":"2024-09-13T16:50:30.613429+0000","last_active":"2024-09-13T16:50:35.093088+0000","last_peered":"2024-09-13T16:50:35.093088+0000","last_clean":"2024-09-13T16:50:35.093088+0000","last_became_active":"2024-09-13T16:50:30.613163+0000","last_became_peered":"2024-09-13T16:50:30.613163+0000","last_unstale":"2024-09-13T16:50:35.093088+0000","last_undegraded":"2024-09-13T16:50:35.093088+0000","last_fullsized":"2024-09-13T16:50:35.093088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456736+0000","last_change":"2024-09-13T16:50:30.607903+0000","last_active":"2024-09-13T16:50:35.456736+0000","last_peered":"2024-09-13T16:50:35.456736+0000","last_clean":"2024-09-13T16:50:35.456736+0000","last_became_active":"2024-09-13T16:50:30.607618+0000","last_became_peered":"2024-09-13T16:50:30.607618+0000","last_unstale":"2024-09-13T16:50:35.456736+0000","last_undegraded":"2024-09-13T16:50:35.456736+0000","last_fullsized":"2024-09-13T16:50:35.456736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091481+0000","last_change":"2024-09-13T16:50:30.603058+0000","last_active":"2024-09-13T16:50:35.091481+0000","last_peered":"2024-09-13T16:50:35.091481+0000","last_clean":"2024-09-13T16:50:35.091481+0000","last_became_active":"2024-09-13T16:50:30.602556+0000","last_became_peered":"2024-09-13T16:50:30.602556+0000","last_unstale":"2024-09-13T16:50:35.091481+0000","last_undegraded":"2024-09-13T16:50:35.091481+0000","last_fullsized":"2024-09-13T16:50:35.091481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456543+0000","last_change":"2024-09-13T16:50:30.599518+0000","last_active":"2024-09-13T16:50:35.456543+0000","last_peered":"2024-09-13T16:50:35.456543+0000","last_clean":"2024-09-13T16:50:35.456543+0000","last_became_active":"2024-09-13T16:50:30.599242+0000","last_became_peered":"2024-09-13T16:50:30.599242+0000","last_unstale":"2024-09-13T16:50:35.456543+0000","last_undegraded":"2024-09-13T16:50:35.456543+0000","last_fullsized":"2024-09-13T16:50:35.456543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091768+0000","last_change":"2024-09-13T16:50:30.610239+0000","last_active":"2024-09-13T16:50:35.091768+0000","last_peered":"2024-09-13T16:50:35.091768+0000","last_clean":"2024-09-13T16:50:35.091768+0000","last_became_active":"2024-09-13T16:50:30.609978+0000","last_became_peered":"2024-09-13T16:50:30.609978+0000","last_unstale":"2024-09-13T16:50:35.091768+0000","last_undegraded":"2024-09-13T16:50:35.091768+0000","last_fullsized":"2024-09-13T16:50:35.091768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:28.715573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462797+0000","last_change":"2024-09-13T16:50:30.618976+0000","last_active":"2024-09-13T16:50:35.462797+0000","last_peered":"2024-09-13T16:50:35.462797+0000","last_clean":"2024-09-13T16:50:35.462797+0000","last_became_active":"2024-09-13T16:50:30.618644+0000","last_became_peered":"2024-09-13T16:50:30.618644+0000","last_unstale":"2024-09-13T16:50:35.462797+0000","last_undegraded":"2024-09-13T16:50:35.462797+0000","last_fullsized":"2024-09-13T16:50:35.462797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:52.041529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092551+0000","last_change":"2024-09-13T16:50:30.609748+0000","last_active":"2024-09-13T16:50:35.092551+0000","last_peered":"2024-09-13T16:50:35.092551+0000","last_clean":"2024-09-13T16:50:35.092551+0000","last_became_active":"2024-09-13T16:50:30.609504+0000","last_became_peered":"2024-09-13T16:50:30.609504+0000","last_unstale":"2024-09-13T16:50:35.092551+0000","last_undegraded":"2024-09-13T16:50:35.092551+0000","last_fullsized":"2024-09-13T16:50:35.092551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500181+0000","last_change":"2024-09-13T16:50:30.618417+0000","last_active":"2024-09-13T16:50:35.500181+0000","last_peered":"2024-09-13T16:50:35.500181+0000","last_clean":"2024-09-13T16:50:35.500181+0000","last_became_active":"2024-09-13T16:50:30.618114+0000","last_became_peered":"2024-09-13T16:50:30.618114+0000","last_unstale":"2024-09-13T16:50:35.500181+0000","last_undegraded":"2024-09-13T16:50:35.500181+0000","last_fullsized":"2024-09-13T16:50:35.500181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:25.758134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.088098+0000","last_change":"2024-09-13T16:50:30.604041+0000","last_active":"2024-09-13T16:50:35.088098+0000","last_peered":"2024-09-13T16:50:35.088098+0000","last_clean":"2024-09-13T16:50:35.088098+0000","last_became_active":"2024-09-13T16:50:30.599336+0000","last_became_peered":"2024-09-13T16:50:30.599336+0000","last_unstale":"2024-09-13T16:50:35.088098+0000","last_undegraded":"2024-09-13T16:50:35.088098+0000","last_fullsized":"2024-09-13T16:50:35.088098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:13.892340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093370+0000","last_change":"2024-09-13T16:50:30.610847+0000","last_active":"2024-09-13T16:50:35.093370+0000","last_peered":"2024-09-13T16:50:35.093370+0000","last_clean":"2024-09-13T16:50:35.093370+0000","last_became_active":"2024-09-13T16:50:30.610581+0000","last_became_peered":"2024-09-13T16:50:30.610581+0000","last_unstale":"2024-09-13T16:50:35.093370+0000","last_undegraded":"2024-09-13T16:50:35.093370+0000","last_fullsized":"2024-09-13T16:50:35.093370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:21.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091388+0000","last_change":"2024-09-13T16:50:30.618144+0000","last_active":"2024-09-13T16:50:35.091388+0000","last_peered":"2024-09-13T16:50:35.091388+0000","last_clean":"2024-09-13T16:50:35.091388+0000","last_became_active":"2024-09-13T16:50:30.617861+0000","last_became_peered":"2024-09-13T16:50:30.617861+0000","last_unstale":"2024-09-13T16:50:35.091388+0000","last_undegraded":"2024-09-13T16:50:35.091388+0000","last_fullsized":"2024-09-13T16:50:35.091388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.400884+0000","last_change":"2024-09-13T16:50:30.590573+0000","last_active":"2024-09-13T16:50:35.400884+0000","last_peered":"2024-09-13T16:50:35.400884+0000","last_clean":"2024-09-13T16:50:35.400884+0000","last_became_active":"2024-09-13T16:50:30.590338+0000","last_became_peered":"2024-09-13T16:50:30.590338+0000","last_unstale":"2024-09-13T16:50:35.400884+0000","last_undegraded":"2024-09-13T16:50:35.400884+0000","last_fullsized":"2024-09-13T16:50:35.400884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462638+0000","last_change":"2024-09-13T16:50:30.615260+0000","last_active":"2024-09-13T16:50:35.462638+0000","last_peered":"2024-09-13T16:50:35.462638+0000","last_clean":"2024-09-13T16:50:35.462638+0000","last_became_active":"2024-09-13T16:50:30.614529+0000","last_became_peered":"2024-09-13T16:50:30.614529+0000","last_unstale":"2024-09-13T16:50:35.462638+0000","last_undegraded":"2024-09-13T16:50:35.462638+0000","last_fullsized":"2024-09-13T16:50:35.462638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.455914+0000","last_change":"2024-09-13T16:50:30.603994+0000","last_active":"2024-09-13T16:50:35.455914+0000","last_peered":"2024-09-13T16:50:35.455914+0000","last_clean":"2024-09-13T16:50:35.455914+0000","last_became_active":"2024-09-13T16:50:30.603610+0000","last_became_peered":"2024-09-13T16:50:30.603610+0000","last_unstale":"2024-09-13T16:50:35.455914+0000","last_undegraded":"2024-09-13T16:50:35.455914+0000","last_fullsized":"2024-09-13T16:50:35.455914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.502132+0000","last_change":"2024-09-13T16:50:30.603873+0000","last_active":"2024-09-13T16:50:35.502132+0000","last_peered":"2024-09-13T16:50:35.502132+0000","last_clean":"2024-09-13T16:50:35.502132+0000","last_became_active":"2024-09-13T16:50:30.603571+0000","last_became_peered":"2024-09-13T16:50:30.603571+0000","last_unstale":"2024-09-13T16:50:35.502132+0000","last_undegraded":"2024-09-13T16:50:35.502132+0000","last_fullsized":"2024-09-13T16:50:35.502132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:22.950313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092808+0000","last_change":"2024-09-13T16:50:30.607038+0000","last_active":"2024-09-13T16:50:35.092808+0000","last_peered":"2024-09-13T16:50:35.092808+0000","last_clean":"2024-09-13T16:50:35.092808+0000","last_became_active":"2024-09-13T16:50:30.606788+0000","last_became_peered":"2024-09-13T16:50:30.606788+0000","last_unstale":"2024-09-13T16:50:35.092808+0000","last_undegraded":"2024-09-13T16:50:35.092808+0000","last_fullsized":"2024-09-13T16:50:35.092808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093030+0000","last_change":"2024-09-13T16:50:30.599427+0000","last_active":"2024-09-13T16:50:35.093030+0000","last_peered":"2024-09-13T16:50:35.093030+0000","last_clean":"2024-09-13T16:50:35.093030+0000","last_became_active":"2024-09-13T16:50:30.599149+0000","last_became_peered":"2024-09-13T16:50:30.599149+0000","last_unstale":"2024-09-13T16:50:35.093030+0000","last_undegraded":"2024-09-13T16:50:35.093030+0000","last_fullsized":"2024-09-13T16:50:35.093030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500732+0000","last_change":"2024-09-13T16:50:30.615710+0000","last_active":"2024-09-13T16:50:35.500732+0000","last_peered":"2024-09-13T16:50:35.500732+0000","last_clean":"2024-09-13T16:50:35.500732+0000","last_became_active":"2024-09-13T16:50:30.615444+0000","last_became_peered":"2024-09-13T16:50:30.615444+0000","last_unstale":"2024-09-13T16:50:35.500732+0000","last_undegraded":"2024-09-13T16:50:35.500732+0000","last_fullsized":"2024-09-13T16:50:35.500732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:35.586398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456590+0000","last_change":"2024-09-13T16:50:30.599700+0000","last_active":"2024-09-13T16:50:35.456590+0000","last_peered":"2024-09-13T16:50:35.456590+0000","last_clean":"2024-09-13T16:50:35.456590+0000","last_became_active":"2024-09-13T16:50:30.599132+0000","last_became_peered":"2024-09-13T16:50:30.599132+0000","last_unstale":"2024-09-13T16:50:35.456590+0000","last_undegraded":"2024-09-13T16:50:35.456590+0000","last_fullsized":"2024-09-13T16:50:35.456590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498763+0000","last_change":"2024-09-13T16:50:30.617544+0000","last_active":"2024-09-13T16:50:35.498763+0000","last_peered":"2024-09-13T16:50:35.498763+0000","last_clean":"2024-09-13T16:50:35.498763+0000","last_became_active":"2024-09-13T16:50:30.617008+0000","last_became_peered":"2024-09-13T16:50:30.617008+0000","last_unstale":"2024-09-13T16:50:35.498763+0000","last_undegraded":"2024-09-13T16:50:35.498763+0000","last_fullsized":"2024-09-13T16:50:35.498763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091720+0000","last_change":"2024-09-13T16:50:30.613899+0000","last_active":"2024-09-13T16:50:35.091720+0000","last_peered":"2024-09-13T16:50:35.091720+0000","last_clean":"2024-09-13T16:50:35.091720+0000","last_became_active":"2024-09-13T16:50:30.613520+0000","last_became_peered":"2024-09-13T16:50:30.613520+0000","last_unstale":"2024-09-13T16:50:35.091720+0000","last_undegraded":"2024-09-13T16:50:35.091720+0000","last_fullsized":"2024-09-13T16:50:35.091720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498780+0000","last_change":"2024-09-13T16:50:30.617169+0000","last_active":"2024-09-13T16:50:35.498780+0000","last_peered":"2024-09-13T16:50:35.498780+0000","last_clean":"2024-09-13T16:50:35.498780+0000","last_became_active":"2024-09-13T16:50:30.616608+0000","last_became_peered":"2024-09-13T16:50:30.616608+0000","last_unstale":"2024-09-13T16:50:35.498780+0000","last_undegraded":"2024-09-13T16:50:35.498780+0000","last_fullsized":"2024-09-13T16:50:35.498780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.117003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092229+0000","last_change":"2024-09-13T16:50:30.601494+0000","last_active":"2024-09-13T16:50:35.092229+0000","last_peered":"2024-09-13T16:50:35.092229+0000","last_clean":"2024-09-13T16:50:35.092229+0000","last_became_active":"2024-09-13T16:50:30.601235+0000","last_became_peered":"2024-09-13T16:50:30.601235+0000","last_unstale":"2024-09-13T16:50:35.092229+0000","last_undegraded":"2024-09-13T16:50:35.092229+0000","last_fullsized":"2024-09-13T16:50:35.092229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:54.244738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463054+0000","last_change":"2024-09-13T16:50:30.611789+0000","last_active":"2024-09-13T16:50:35.463054+0000","last_peered":"2024-09-13T16:50:35.463054+0000","last_clean":"2024-09-13T16:50:35.463054+0000","last_became_active":"2024-09-13T16:50:30.608646+0000","last_became_peered":"2024-09-13T16:50:30.608646+0000","last_unstale":"2024-09-13T16:50:35.463054+0000","last_undegraded":"2024-09-13T16:50:35.463054+0000","last_fullsized":"2024-09-13T16:50:35.463054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.145790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402335+0000","last_change":"2024-09-13T16:50:30.609892+0000","last_active":"2024-09-13T16:50:35.402335+0000","last_peered":"2024-09-13T16:50:35.402335+0000","last_clean":"2024-09-13T16:50:35.402335+0000","last_became_active":"2024-09-13T16:50:30.609664+0000","last_became_peered":"2024-09-13T16:50:30.609664+0000","last_unstale":"2024-09-13T16:50:35.402335+0000","last_undegraded":"2024-09-13T16:50:35.402335+0000","last_fullsized":"2024-09-13T16:50:35.402335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500628+0000","last_change":"2024-09-13T16:50:30.607977+0000","last_active":"2024-09-13T16:50:35.500628+0000","last_peered":"2024-09-13T16:50:35.500628+0000","last_clean":"2024-09-13T16:50:35.500628+0000","last_became_active":"2024-09-13T16:50:30.607735+0000","last_became_peered":"2024-09-13T16:50:30.607735+0000","last_unstale":"2024-09-13T16:50:35.500628+0000","last_undegraded":"2024-09-13T16:50:35.500628+0000","last_fullsized":"2024-09-13T16:50:35.500628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:21.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":864256,"data_stored":697795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.631}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5780000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":278528,"data_stored":106971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":106845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":697851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:51:03.825 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:51:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.021+0000 7f6344c00700 1 Processor -- start 2024-09-13T16:51:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.021+0000 7f6344c00700 1 -- start start 2024-09-13T16:51:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f6344c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63401443b0 0x7f63400b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f6344c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f634006ccf0 0x7f634015b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f6344c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63400ce8e0 con 0x7f6340144790 2024-09-13T16:51:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f6344c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63400ccd30 con 0x7f634006ccf0 2024-09-13T16:51:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f6344c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63400cd010 con 0x7f63401443b0 2024-09-13T16:51:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f6337e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6340144790 0x7f634006c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34610/0 (socket says 172.21.15.3:34610) 2024-09-13T16:51:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f6337e00700 1 -- 172.21.15.3:0/2998613425 learned_addr learned my addr 172.21.15.3:0/2998613425 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f633fe00700 1 --2- 172.21.15.3:0/2998613425 >> v2:172.21.15.3:3300/0 conn(0x7f634006ccf0 0x7f634015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.025+0000 7f633f400700 1 --2- 172.21.15.3:0/2998613425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63401443b0 0x7f63400b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633fe00700 1 -- 172.21.15.3:0/2998613425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63401443b0 msgr2=0x7f63400b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633fe00700 1 --2- 172.21.15.3:0/2998613425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63401443b0 0x7f63400b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633fe00700 1 -- 172.21.15.3:0/2998613425 >> v1:172.21.15.3:6789/0 conn(0x7f6340144790 legacy=0x7f634006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633ea00700 1 -- 172.21.15.3:0/2998613425 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379964113 0 0) 0x7f63400ce8e0 con 0x7f6340144790 2024-09-13T16:51:04.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633fe00700 1 -- 172.21.15.3:0/2998613425 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63400cd2f0 con 0x7f634006ccf0 2024-09-13T16:51:04.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633fe00700 1 --2- 172.21.15.3:0/2998613425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015b220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6338009d90 tx=0x7f633800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadd638cadf36f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633ea00700 1 -- 172.21.15.3:0/2998613425 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633800e070 con 0x7f634006ccf0 2024-09-13T16:51:04.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633ea00700 1 -- 172.21.15.3:0/2998613425 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633800bb20 con 0x7f634006ccf0 2024-09-13T16:51:04.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f633ea00700 1 -- 172.21.15.3:0/2998613425 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338012430 con 0x7f634006ccf0 2024-09-13T16:51:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f6344c00700 1 -- 172.21.15.3:0/2998613425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 msgr2=0x7f634015b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f6344c00700 1 --2- 172.21.15.3:0/2998613425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015b220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6338009d90 tx=0x7f633800b330 comp rx=0 tx=0).stop 2024-09-13T16:51:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f6344c00700 1 -- 172.21.15.3:0/2998613425 shutdown_connections 2024-09-13T16:51:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f6344c00700 1 --2- 172.21.15.3:0/2998613425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015b220 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f6344c00700 1 --2- 172.21.15.3:0/2998613425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63401443b0 0x7f63400b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f6344c00700 1 -- 172.21.15.3:0/2998613425 >> 172.21.15.3:0/2998613425 conn(0x7f63400b0cf0 msgr2=0x7f63400b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f6344c00700 1 -- 172.21.15.3:0/2998613425 shutdown_connections 2024-09-13T16:51:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.029+0000 7f6344c00700 1 -- 172.21.15.3:0/2998613425 wait complete. 2024-09-13T16:51:04.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f6344c00700 1 Processor -- start 2024-09-13T16:51:04.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f6344c00700 1 -- start start 2024-09-13T16:51:04.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f6344c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f6344c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63401443b0 0x7f6340138a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f6344c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6340138f90 0x7f6340142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f6344c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63401381b0 con 0x7f634006ccf0 2024-09-13T16:51:04.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f6344c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63400a86d0 con 0x7f63401443b0 2024-09-13T16:51:04.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f6344c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6340137910 con 0x7f6340138f90 2024-09-13T16:51:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f633f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f633f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32862/0 (socket says 172.21.15.3:32862) 2024-09-13T16:51:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f633f400700 1 -- 172.21.15.3:0/3916620543 learned_addr learned my addr 172.21.15.3:0/3916620543 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.033+0000 7f633fe00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6340138f90 0x7f6340142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f6337e00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63401443b0 0x7f6340138a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f633f400700 1 -- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6340138f90 msgr2=0x7f6340142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f633f400700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6340138f90 0x7f6340142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f633f400700 1 -- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63401443b0 msgr2=0x7f6340138a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f633f400700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63401443b0 0x7f6340138a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f633f400700 1 -- 172.21.15.3:0/3916620543 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338009a70 con 0x7f634006ccf0 2024-09-13T16:51:04.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f633fe00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6340138f90 0x7f6340142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:04.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f633f400700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015a630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f633000ea00 tx=0x7f633000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f6337400700 1 -- 172.21.15.3:0/3916620543 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330009720 con 0x7f634006ccf0 2024-09-13T16:51:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f6337400700 1 -- 172.21.15.3:0/3916620543 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633000c850 con 0x7f634006ccf0 2024-09-13T16:51:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6340137bf0 con 0x7f634006ccf0 2024-09-13T16:51:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f6337400700 1 -- 172.21.15.3:0/3916620543 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633000cbc0 con 0x7f634006ccf0 2024-09-13T16:51:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634015ab70 con 0x7f634006ccf0 2024-09-13T16:51:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.037+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63400b39f0 con 0x7f634006ccf0 2024-09-13T16:51:04.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.041+0000 7f6337400700 1 -- 172.21.15.3:0/3916620543 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f633000cd20 con 0x7f634006ccf0 2024-09-13T16:51:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.041+0000 7f6337400700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f631c066760 0x7f631c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.041+0000 7f6337400700 1 -- 172.21.15.3:0/3916620543 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6330084650 con 0x7f634006ccf0 2024-09-13T16:51:04.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.041+0000 7f6337e00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f631c066760 0x7f631c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.045+0000 7f6337e00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f631c066760 0x7f631c068c20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63401439e0 tx=0x7f6328007cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.049+0000 7f6337400700 1 -- 172.21.15.3:0/3916620543 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6330051630 con 0x7f634006ccf0 2024-09-13T16:51:04.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.225+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63400b2ac0 con 0x7f631c066760 2024-09-13T16:51:04.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.249+0000 7f6337400700 1 -- 172.21.15.3:0/3916620543 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345056 (secure 0 0 0) 0x7f63400b2ac0 con 0x7f631c066760 2024-09-13T16:51:04.260 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f631c066760 msgr2=0x7f631c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f631c066760 0x7f631c068c20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63401439e0 tx=0x7f6328007cb0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 msgr2=0x7f634015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015a630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f633000ea00 tx=0x7f633000edc0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 shutdown_connections 2024-09-13T16:51:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f631c066760 0x7f631c068c20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6340138f90 0x7f6340142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63401443b0 0x7f6340138a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 --2- 172.21.15.3:0/3916620543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634006ccf0 0x7f634015a630 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 >> 172.21.15.3:0/3916620543 conn(0x7f63400b0cf0 msgr2=0x7f634013f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 shutdown_connections 2024-09-13T16:51:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.257+0000 7f6344c00700 1 -- 172.21.15.3:0/3916620543 wait complete. 2024-09-13T16:51:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:51:04.283 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-13T16:51:03.205099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227664,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754747056,"statfs":{"total":773094113280,"available":772860985344,"internally_reserved":0,"allocated":3383296,"data_stored":2044682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006306"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463496+0000","last_change":"2024-09-13T16:50:30.106124+0000","last_active":"2024-09-13T16:50:35.463496+0000","last_peered":"2024-09-13T16:50:35.463496+0000","last_clean":"2024-09-13T16:50:35.463496+0000","last_became_active":"2024-09-13T16:50:30.105874+0000","last_became_peered":"2024-09-13T16:50:30.105874+0000","last_unstale":"2024-09-13T16:50:35.463496+0000","last_undegraded":"2024-09-13T16:50:35.463496+0000","last_fullsized":"2024-09-13T16:50:35.463496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:38.882189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456272+0000","last_change":"2024-09-13T16:50:30.598817+0000","last_active":"2024-09-13T16:50:35.456272+0000","last_peered":"2024-09-13T16:50:35.456272+0000","last_clean":"2024-09-13T16:50:35.456272+0000","last_became_active":"2024-09-13T16:50:30.598483+0000","last_became_peered":"2024-09-13T16:50:30.598483+0000","last_unstale":"2024-09-13T16:50:35.456272+0000","last_undegraded":"2024-09-13T16:50:35.456272+0000","last_fullsized":"2024-09-13T16:50:35.456272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.511304+0000","last_change":"2024-09-13T16:50:35.507978+0000","last_active":"2024-09-13T16:50:35.511304+0000","last_peered":"2024-09-13T16:50:35.511304+0000","last_clean":"2024-09-13T16:50:35.511304+0000","last_became_active":"2024-09-13T16:50:35.506655+0000","last_became_peered":"2024-09-13T16:50:35.506655+0000","last_unstale":"2024-09-13T16:50:35.511304+0000","last_undegraded":"2024-09-13T16:50:35.511304+0000","last_fullsized":"2024-09-13T16:50:35.511304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:29.062934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456822+0000","last_change":"2024-09-13T16:50:30.101330+0000","last_active":"2024-09-13T16:50:35.456822+0000","last_peered":"2024-09-13T16:50:35.456822+0000","last_clean":"2024-09-13T16:50:35.456822+0000","last_became_active":"2024-09-13T16:50:30.101038+0000","last_became_peered":"2024-09-13T16:50:30.101038+0000","last_unstale":"2024-09-13T16:50:35.456822+0000","last_undegraded":"2024-09-13T16:50:35.456822+0000","last_fullsized":"2024-09-13T16:50:35.456822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463263+0000","last_change":"2024-09-13T16:50:30.609337+0000","last_active":"2024-09-13T16:50:35.463263+0000","last_peered":"2024-09-13T16:50:35.463263+0000","last_clean":"2024-09-13T16:50:35.463263+0000","last_became_active":"2024-09-13T16:50:30.609098+0000","last_became_peered":"2024-09-13T16:50:30.609098+0000","last_unstale":"2024-09-13T16:50:35.463263+0000","last_undegraded":"2024-09-13T16:50:35.463263+0000","last_fullsized":"2024-09-13T16:50:35.463263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498079+0000","last_change":"2024-09-13T16:50:34.633734+0000","last_active":"2024-09-13T16:50:35.498079+0000","last_peered":"2024-09-13T16:50:35.498079+0000","last_clean":"2024-09-13T16:50:35.498079+0000","last_became_active":"2024-09-13T16:50:34.632931+0000","last_became_peered":"2024-09-13T16:50:34.632931+0000","last_unstale":"2024-09-13T16:50:35.498079+0000","last_undegraded":"2024-09-13T16:50:35.498079+0000","last_fullsized":"2024-09-13T16:50:35.498079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:37.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500017+0000","last_change":"2024-09-13T16:50:30.108543+0000","last_active":"2024-09-13T16:50:35.500017+0000","last_peered":"2024-09-13T16:50:35.500017+0000","last_clean":"2024-09-13T16:50:35.500017+0000","last_became_active":"2024-09-13T16:50:30.108282+0000","last_became_peered":"2024-09-13T16:50:30.108282+0000","last_unstale":"2024-09-13T16:50:35.500017+0000","last_undegraded":"2024-09-13T16:50:35.500017+0000","last_fullsized":"2024-09-13T16:50:35.500017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092867+0000","last_change":"2024-09-13T16:50:30.594489+0000","last_active":"2024-09-13T16:50:35.092867+0000","last_peered":"2024-09-13T16:50:35.092867+0000","last_clean":"2024-09-13T16:50:35.092867+0000","last_became_active":"2024-09-13T16:50:30.593342+0000","last_became_peered":"2024-09-13T16:50:30.593342+0000","last_unstale":"2024-09-13T16:50:35.092867+0000","last_undegraded":"2024-09-13T16:50:35.092867+0000","last_fullsized":"2024-09-13T16:50:35.092867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500758+0000","last_change":"2024-09-13T16:50:34.633900+0000","last_active":"2024-09-13T16:50:35.500758+0000","last_peered":"2024-09-13T16:50:35.500758+0000","last_clean":"2024-09-13T16:50:35.500758+0000","last_became_active":"2024-09-13T16:50:34.633428+0000","last_became_peered":"2024-09-13T16:50:34.633428+0000","last_unstale":"2024-09-13T16:50:35.500758+0000","last_undegraded":"2024-09-13T16:50:35.500758+0000","last_fullsized":"2024-09-13T16:50:35.500758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:01.459147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499214+0000","last_change":"2024-09-13T16:50:30.119534+0000","last_active":"2024-09-13T16:50:35.499214+0000","last_peered":"2024-09-13T16:50:35.499214+0000","last_clean":"2024-09-13T16:50:35.499214+0000","last_became_active":"2024-09-13T16:50:30.119177+0000","last_became_peered":"2024-09-13T16:50:30.119177+0000","last_unstale":"2024-09-13T16:50:35.499214+0000","last_undegraded":"2024-09-13T16:50:35.499214+0000","last_fullsized":"2024-09-13T16:50:35.499214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092738+0000","last_change":"2024-09-13T16:50:30.604539+0000","last_active":"2024-09-13T16:50:35.092738+0000","last_peered":"2024-09-13T16:50:35.092738+0000","last_clean":"2024-09-13T16:50:35.092738+0000","last_became_active":"2024-09-13T16:50:30.604192+0000","last_became_peered":"2024-09-13T16:50:30.604192+0000","last_unstale":"2024-09-13T16:50:35.092738+0000","last_undegraded":"2024-09-13T16:50:35.092738+0000","last_fullsized":"2024-09-13T16:50:35.092738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500774+0000","last_change":"2024-09-13T16:50:34.632525+0000","last_active":"2024-09-13T16:50:35.500774+0000","last_peered":"2024-09-13T16:50:35.500774+0000","last_clean":"2024-09-13T16:50:35.500774+0000","last_became_active":"2024-09-13T16:50:34.632114+0000","last_became_peered":"2024-09-13T16:50:34.632114+0000","last_unstale":"2024-09-13T16:50:35.500774+0000","last_undegraded":"2024-09-13T16:50:35.500774+0000","last_fullsized":"2024-09-13T16:50:35.500774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091628+0000","last_change":"2024-09-13T16:50:30.611124+0000","last_active":"2024-09-13T16:50:35.091628+0000","last_peered":"2024-09-13T16:50:35.091628+0000","last_clean":"2024-09-13T16:50:35.091628+0000","last_became_active":"2024-09-13T16:50:30.607850+0000","last_became_peered":"2024-09-13T16:50:30.607850+0000","last_unstale":"2024-09-13T16:50:35.091628+0000","last_undegraded":"2024-09-13T16:50:35.091628+0000","last_fullsized":"2024-09-13T16:50:35.091628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499169+0000","last_change":"2024-09-13T16:50:30.106841+0000","last_active":"2024-09-13T16:50:35.499169+0000","last_peered":"2024-09-13T16:50:35.499169+0000","last_clean":"2024-09-13T16:50:35.499169+0000","last_became_active":"2024-09-13T16:50:30.106581+0000","last_became_peered":"2024-09-13T16:50:30.106581+0000","last_unstale":"2024-09-13T16:50:35.499169+0000","last_undegraded":"2024-09-13T16:50:35.499169+0000","last_fullsized":"2024-09-13T16:50:35.499169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501399+0000","last_change":"2024-09-13T16:50:34.633477+0000","last_active":"2024-09-13T16:50:35.501399+0000","last_peered":"2024-09-13T16:50:35.501399+0000","last_clean":"2024-09-13T16:50:35.501399+0000","last_became_active":"2024-09-13T16:50:34.633072+0000","last_became_peered":"2024-09-13T16:50:34.633072+0000","last_unstale":"2024-09-13T16:50:35.501399+0000","last_undegraded":"2024-09-13T16:50:35.501399+0000","last_fullsized":"2024-09-13T16:50:35.501399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:14.360209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456436+0000","last_change":"2024-09-13T16:50:30.611139+0000","last_active":"2024-09-13T16:50:35.456436+0000","last_peered":"2024-09-13T16:50:35.456436+0000","last_clean":"2024-09-13T16:50:35.456436+0000","last_became_active":"2024-09-13T16:50:30.610455+0000","last_became_peered":"2024-09-13T16:50:30.610455+0000","last_unstale":"2024-09-13T16:50:35.456436+0000","last_undegraded":"2024-09-13T16:50:35.456436+0000","last_fullsized":"2024-09-13T16:50:35.456436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402349+0000","last_change":"2024-09-13T16:50:30.113719+0000","last_active":"2024-09-13T16:50:35.402349+0000","last_peered":"2024-09-13T16:50:35.402349+0000","last_clean":"2024-09-13T16:50:35.402349+0000","last_became_active":"2024-09-13T16:50:30.111437+0000","last_became_peered":"2024-09-13T16:50:30.111437+0000","last_unstale":"2024-09-13T16:50:35.402349+0000","last_undegraded":"2024-09-13T16:50:35.402349+0000","last_fullsized":"2024-09-13T16:50:35.402349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501838+0000","last_change":"2024-09-13T16:50:34.638895+0000","last_active":"2024-09-13T16:50:35.501838+0000","last_peered":"2024-09-13T16:50:35.501838+0000","last_clean":"2024-09-13T16:50:35.501838+0000","last_became_active":"2024-09-13T16:50:34.638431+0000","last_became_peered":"2024-09-13T16:50:34.638431+0000","last_unstale":"2024-09-13T16:50:35.501838+0000","last_undegraded":"2024-09-13T16:50:35.501838+0000","last_fullsized":"2024-09-13T16:50:35.501838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:51.705201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.457086+0000","last_change":"2024-09-13T16:50:30.599950+0000","last_active":"2024-09-13T16:50:35.457086+0000","last_peered":"2024-09-13T16:50:35.457086+0000","last_clean":"2024-09-13T16:50:35.457086+0000","last_became_active":"2024-09-13T16:50:30.599401+0000","last_became_peered":"2024-09-13T16:50:30.599401+0000","last_unstale":"2024-09-13T16:50:35.457086+0000","last_undegraded":"2024-09-13T16:50:35.457086+0000","last_fullsized":"2024-09-13T16:50:35.457086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401297+0000","last_change":"2024-09-13T16:50:30.110698+0000","last_active":"2024-09-13T16:50:35.401297+0000","last_peered":"2024-09-13T16:50:35.401297+0000","last_clean":"2024-09-13T16:50:35.401297+0000","last_became_active":"2024-09-13T16:50:30.110343+0000","last_became_peered":"2024-09-13T16:50:30.110343+0000","last_unstale":"2024-09-13T16:50:35.401297+0000","last_undegraded":"2024-09-13T16:50:35.401297+0000","last_fullsized":"2024-09-13T16:50:35.401297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.512080+0000","last_change":"2024-09-13T16:50:35.510605+0000","last_active":"2024-09-13T16:50:35.512080+0000","last_peered":"2024-09-13T16:50:35.512080+0000","last_clean":"2024-09-13T16:50:35.512080+0000","last_became_active":"2024-09-13T16:50:35.510230+0000","last_became_peered":"2024-09-13T16:50:35.510230+0000","last_unstale":"2024-09-13T16:50:35.512080+0000","last_undegraded":"2024-09-13T16:50:35.512080+0000","last_fullsized":"2024-09-13T16:50:35.512080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500916+0000","last_change":"2024-09-13T16:50:30.608538+0000","last_active":"2024-09-13T16:50:35.500916+0000","last_peered":"2024-09-13T16:50:35.500916+0000","last_clean":"2024-09-13T16:50:35.500916+0000","last_became_active":"2024-09-13T16:50:30.608195+0000","last_became_peered":"2024-09-13T16:50:30.608195+0000","last_unstale":"2024-09-13T16:50:35.500916+0000","last_undegraded":"2024-09-13T16:50:35.500916+0000","last_fullsized":"2024-09-13T16:50:35.500916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:48.431406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.497470+0000","last_change":"2024-09-13T16:50:30.110135+0000","last_active":"2024-09-13T16:50:35.497470+0000","last_peered":"2024-09-13T16:50:35.497470+0000","last_clean":"2024-09-13T16:50:35.497470+0000","last_became_active":"2024-09-13T16:50:30.109520+0000","last_became_peered":"2024-09-13T16:50:30.109520+0000","last_unstale":"2024-09-13T16:50:35.497470+0000","last_undegraded":"2024-09-13T16:50:35.497470+0000","last_fullsized":"2024-09-13T16:50:35.497470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501389+0000","last_change":"2024-09-13T16:50:34.635782+0000","last_active":"2024-09-13T16:50:35.501389+0000","last_peered":"2024-09-13T16:50:35.501389+0000","last_clean":"2024-09-13T16:50:35.501389+0000","last_became_active":"2024-09-13T16:50:34.635349+0000","last_became_peered":"2024-09-13T16:50:34.635349+0000","last_unstale":"2024-09-13T16:50:35.501389+0000","last_undegraded":"2024-09-13T16:50:35.501389+0000","last_fullsized":"2024-09-13T16:50:35.501389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:58.754567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463122+0000","last_change":"2024-09-13T16:50:30.614804+0000","last_active":"2024-09-13T16:50:35.463122+0000","last_peered":"2024-09-13T16:50:35.463122+0000","last_clean":"2024-09-13T16:50:35.463122+0000","last_became_active":"2024-09-13T16:50:30.614458+0000","last_became_peered":"2024-09-13T16:50:30.614458+0000","last_unstale":"2024-09-13T16:50:35.463122+0000","last_undegraded":"2024-09-13T16:50:35.463122+0000","last_fullsized":"2024-09-13T16:50:35.463122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093301+0000","last_change":"2024-09-13T16:50:30.107253+0000","last_active":"2024-09-13T16:50:35.093301+0000","last_peered":"2024-09-13T16:50:35.093301+0000","last_clean":"2024-09-13T16:50:35.093301+0000","last_became_active":"2024-09-13T16:50:30.106448+0000","last_became_peered":"2024-09-13T16:50:30.106448+0000","last_unstale":"2024-09-13T16:50:35.093301+0000","last_undegraded":"2024-09-13T16:50:35.093301+0000","last_fullsized":"2024-09-13T16:50:35.093301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:12.880879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.507157+0000","last_change":"2024-09-13T16:50:35.504918+0000","last_active":"2024-09-13T16:50:35.507157+0000","last_peered":"2024-09-13T16:50:35.507157+0000","last_clean":"2024-09-13T16:50:35.507157+0000","last_became_active":"2024-09-13T16:50:35.504252+0000","last_became_peered":"2024-09-13T16:50:35.504252+0000","last_unstale":"2024-09-13T16:50:35.507157+0000","last_undegraded":"2024-09-13T16:50:35.507157+0000","last_fullsized":"2024-09-13T16:50:35.507157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:54.702371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456826+0000","last_change":"2024-09-13T16:50:30.610870+0000","last_active":"2024-09-13T16:50:35.456826+0000","last_peered":"2024-09-13T16:50:35.456826+0000","last_clean":"2024-09-13T16:50:35.456826+0000","last_became_active":"2024-09-13T16:50:30.610473+0000","last_became_peered":"2024-09-13T16:50:30.610473+0000","last_unstale":"2024-09-13T16:50:35.456826+0000","last_undegraded":"2024-09-13T16:50:35.456826+0000","last_fullsized":"2024-09-13T16:50:35.456826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.940724+0000","last_change":"2024-09-13T16:50:30.111250+0000","last_active":"2024-09-13T16:50:35.940724+0000","last_peered":"2024-09-13T16:50:35.940724+0000","last_clean":"2024-09-13T16:50:35.940724+0000","last_became_active":"2024-09-13T16:50:30.109959+0000","last_became_peered":"2024-09-13T16:50:30.109959+0000","last_unstale":"2024-09-13T16:50:35.940724+0000","last_undegraded":"2024-09-13T16:50:35.940724+0000","last_fullsized":"2024-09-13T16:50:35.940724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509887+0000","last_change":"2024-09-13T16:50:35.501491+0000","last_active":"2024-09-13T16:50:35.509887+0000","last_peered":"2024-09-13T16:50:35.509887+0000","last_clean":"2024-09-13T16:50:35.509887+0000","last_became_active":"2024-09-13T16:50:35.500108+0000","last_became_peered":"2024-09-13T16:50:35.500108+0000","last_unstale":"2024-09-13T16:50:35.509887+0000","last_undegraded":"2024-09-13T16:50:35.509887+0000","last_fullsized":"2024-09-13T16:50:35.509887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:30.666178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401804+0000","last_change":"2024-09-13T16:50:30.606888+0000","last_active":"2024-09-13T16:50:35.401804+0000","last_peered":"2024-09-13T16:50:35.401804+0000","last_clean":"2024-09-13T16:50:35.401804+0000","last_became_active":"2024-09-13T16:50:30.605360+0000","last_became_peered":"2024-09-13T16:50:30.605360+0000","last_unstale":"2024-09-13T16:50:35.401804+0000","last_undegraded":"2024-09-13T16:50:35.401804+0000","last_fullsized":"2024-09-13T16:50:35.401804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500383+0000","last_change":"2024-09-13T16:50:30.111119+0000","last_active":"2024-09-13T16:50:35.500383+0000","last_peered":"2024-09-13T16:50:35.500383+0000","last_clean":"2024-09-13T16:50:35.500383+0000","last_became_active":"2024-09-13T16:50:30.110892+0000","last_became_peered":"2024-09-13T16:50:30.110892+0000","last_unstale":"2024-09-13T16:50:35.500383+0000","last_undegraded":"2024-09-13T16:50:35.500383+0000","last_fullsized":"2024-09-13T16:50:35.500383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509685+0000","last_change":"2024-09-13T16:50:35.505149+0000","last_active":"2024-09-13T16:50:35.509685+0000","last_peered":"2024-09-13T16:50:35.509685+0000","last_clean":"2024-09-13T16:50:35.509685+0000","last_became_active":"2024-09-13T16:50:35.502086+0000","last_became_peered":"2024-09-13T16:50:35.502086+0000","last_unstale":"2024-09-13T16:50:35.509685+0000","last_undegraded":"2024-09-13T16:50:35.509685+0000","last_fullsized":"2024-09-13T16:50:35.509685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:08.212857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092835+0000","last_change":"2024-09-13T16:50:30.601700+0000","last_active":"2024-09-13T16:50:35.092835+0000","last_peered":"2024-09-13T16:50:35.092835+0000","last_clean":"2024-09-13T16:50:35.092835+0000","last_became_active":"2024-09-13T16:50:30.601432+0000","last_became_peered":"2024-09-13T16:50:30.601432+0000","last_unstale":"2024-09-13T16:50:35.092835+0000","last_undegraded":"2024-09-13T16:50:35.092835+0000","last_fullsized":"2024-09-13T16:50:35.092835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:25.758134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402007+0000","last_change":"2024-09-13T16:50:30.112636+0000","last_active":"2024-09-13T16:50:35.402007+0000","last_peered":"2024-09-13T16:50:35.402007+0000","last_clean":"2024-09-13T16:50:35.402007+0000","last_became_active":"2024-09-13T16:50:30.112121+0000","last_became_peered":"2024-09-13T16:50:30.112121+0000","last_unstale":"2024-09-13T16:50:35.402007+0000","last_undegraded":"2024-09-13T16:50:35.402007+0000","last_fullsized":"2024-09-13T16:50:35.402007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401192+0000","last_change":"2024-09-13T16:50:34.633860+0000","last_active":"2024-09-13T16:50:35.401192+0000","last_peered":"2024-09-13T16:50:35.401192+0000","last_clean":"2024-09-13T16:50:35.401192+0000","last_became_active":"2024-09-13T16:50:34.633435+0000","last_became_peered":"2024-09-13T16:50:34.633435+0000","last_unstale":"2024-09-13T16:50:35.401192+0000","last_undegraded":"2024-09-13T16:50:35.401192+0000","last_fullsized":"2024-09-13T16:50:35.401192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:27.883765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501232+0000","last_change":"2024-09-13T16:50:30.613267+0000","last_active":"2024-09-13T16:50:35.501232+0000","last_peered":"2024-09-13T16:50:35.501232+0000","last_clean":"2024-09-13T16:50:35.501232+0000","last_became_active":"2024-09-13T16:50:30.612861+0000","last_became_peered":"2024-09-13T16:50:30.612861+0000","last_unstale":"2024-09-13T16:50:35.501232+0000","last_undegraded":"2024-09-13T16:50:35.501232+0000","last_fullsized":"2024-09-13T16:50:35.501232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:27.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091946+0000","last_change":"2024-09-13T16:50:30.110348+0000","last_active":"2024-09-13T16:50:35.091946+0000","last_peered":"2024-09-13T16:50:35.091946+0000","last_clean":"2024-09-13T16:50:35.091946+0000","last_became_active":"2024-09-13T16:50:30.110041+0000","last_became_peered":"2024-09-13T16:50:30.110041+0000","last_unstale":"2024-09-13T16:50:35.091946+0000","last_undegraded":"2024-09-13T16:50:35.091946+0000","last_fullsized":"2024-09-13T16:50:35.091946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401674+0000","last_change":"2024-09-13T16:50:34.633493+0000","last_active":"2024-09-13T16:50:35.401674+0000","last_peered":"2024-09-13T16:50:35.401674+0000","last_clean":"2024-09-13T16:50:35.401674+0000","last_became_active":"2024-09-13T16:50:34.632442+0000","last_became_peered":"2024-09-13T16:50:34.632442+0000","last_unstale":"2024-09-13T16:50:35.401674+0000","last_undegraded":"2024-09-13T16:50:35.401674+0000","last_fullsized":"2024-09-13T16:50:35.401674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401509+0000","last_change":"2024-09-13T16:50:30.590118+0000","last_active":"2024-09-13T16:50:35.401509+0000","last_peered":"2024-09-13T16:50:35.401509+0000","last_clean":"2024-09-13T16:50:35.401509+0000","last_became_active":"2024-09-13T16:50:30.589880+0000","last_became_peered":"2024-09-13T16:50:30.589880+0000","last_unstale":"2024-09-13T16:50:35.401509+0000","last_undegraded":"2024-09-13T16:50:35.401509+0000","last_fullsized":"2024-09-13T16:50:35.401509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:32.908616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087743+0000","last_change":"2024-09-13T16:50:30.103427+0000","last_active":"2024-09-13T16:50:35.087743+0000","last_peered":"2024-09-13T16:50:35.087743+0000","last_clean":"2024-09-13T16:50:35.087743+0000","last_became_active":"2024-09-13T16:50:30.102964+0000","last_became_peered":"2024-09-13T16:50:30.102964+0000","last_unstale":"2024-09-13T16:50:35.087743+0000","last_undegraded":"2024-09-13T16:50:35.087743+0000","last_fullsized":"2024-09-13T16:50:35.087743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402109+0000","last_change":"2024-09-13T16:50:34.632823+0000","last_active":"2024-09-13T16:50:35.402109+0000","last_peered":"2024-09-13T16:50:35.402109+0000","last_clean":"2024-09-13T16:50:35.402109+0000","last_became_active":"2024-09-13T16:50:34.632261+0000","last_became_peered":"2024-09-13T16:50:34.632261+0000","last_unstale":"2024-09-13T16:50:35.402109+0000","last_undegraded":"2024-09-13T16:50:35.402109+0000","last_fullsized":"2024-09-13T16:50:35.402109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462419+0000","last_change":"2024-09-13T16:50:30.606638+0000","last_active":"2024-09-13T16:50:35.462419+0000","last_peered":"2024-09-13T16:50:35.462419+0000","last_clean":"2024-09-13T16:50:35.462419+0000","last_became_active":"2024-09-13T16:50:30.606375+0000","last_became_peered":"2024-09-13T16:50:30.606375+0000","last_unstale":"2024-09-13T16:50:35.462419+0000","last_undegraded":"2024-09-13T16:50:35.462419+0000","last_fullsized":"2024-09-13T16:50:35.462419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:27.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498015+0000","last_change":"2024-09-13T16:50:30.115629+0000","last_active":"2024-09-13T16:50:35.498015+0000","last_peered":"2024-09-13T16:50:35.498015+0000","last_clean":"2024-09-13T16:50:35.498015+0000","last_became_active":"2024-09-13T16:50:30.115336+0000","last_became_peered":"2024-09-13T16:50:30.115336+0000","last_unstale":"2024-09-13T16:50:35.498015+0000","last_undegraded":"2024-09-13T16:50:35.498015+0000","last_fullsized":"2024-09-13T16:50:35.498015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:47.419945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509728+0000","last_change":"2024-09-13T16:50:35.507866+0000","last_active":"2024-09-13T16:50:35.509728+0000","last_peered":"2024-09-13T16:50:35.509728+0000","last_clean":"2024-09-13T16:50:35.509728+0000","last_became_active":"2024-09-13T16:50:35.505548+0000","last_became_peered":"2024-09-13T16:50:35.505548+0000","last_unstale":"2024-09-13T16:50:35.509728+0000","last_undegraded":"2024-09-13T16:50:35.509728+0000","last_fullsized":"2024-09-13T16:50:35.509728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.455646+0000","last_change":"2024-09-13T16:50:30.596915+0000","last_active":"2024-09-13T16:50:35.455646+0000","last_peered":"2024-09-13T16:50:35.455646+0000","last_clean":"2024-09-13T16:50:35.455646+0000","last_became_active":"2024-09-13T16:50:30.596643+0000","last_became_peered":"2024-09-13T16:50:30.596643+0000","last_unstale":"2024-09-13T16:50:35.455646+0000","last_undegraded":"2024-09-13T16:50:35.455646+0000","last_fullsized":"2024-09-13T16:50:35.455646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092229+0000","last_change":"2024-09-13T16:50:30.115172+0000","last_active":"2024-09-13T16:50:35.092229+0000","last_peered":"2024-09-13T16:50:35.092229+0000","last_clean":"2024-09-13T16:50:35.092229+0000","last_became_active":"2024-09-13T16:50:30.114633+0000","last_became_peered":"2024-09-13T16:50:30.114633+0000","last_unstale":"2024-09-13T16:50:35.092229+0000","last_undegraded":"2024-09-13T16:50:35.092229+0000","last_fullsized":"2024-09-13T16:50:35.092229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498080+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:50:35.498080+0000","last_peered":"2024-09-13T16:50:35.498080+0000","last_clean":"2024-09-13T16:50:35.498080+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:50:35.498080+0000","last_undegraded":"2024-09-13T16:50:35.498080+0000","last_fullsized":"2024-09-13T16:50:35.498080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.506121+0000","last_change":"2024-09-13T16:50:35.502219+0000","last_active":"2024-09-13T16:50:35.506121+0000","last_peered":"2024-09-13T16:50:35.506121+0000","last_clean":"2024-09-13T16:50:35.506121+0000","last_became_active":"2024-09-13T16:50:35.501877+0000","last_became_peered":"2024-09-13T16:50:35.501877+0000","last_unstale":"2024-09-13T16:50:35.506121+0000","last_undegraded":"2024-09-13T16:50:35.506121+0000","last_fullsized":"2024-09-13T16:50:35.506121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:37.440533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500614+0000","last_change":"2024-09-13T16:50:30.604143+0000","last_active":"2024-09-13T16:50:35.500614+0000","last_peered":"2024-09-13T16:50:35.500614+0000","last_clean":"2024-09-13T16:50:35.500614+0000","last_became_active":"2024-09-13T16:50:30.603882+0000","last_became_peered":"2024-09-13T16:50:30.603882+0000","last_unstale":"2024-09-13T16:50:35.500614+0000","last_undegraded":"2024-09-13T16:50:35.500614+0000","last_fullsized":"2024-09-13T16:50:35.500614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:31.996725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087857+0000","last_change":"2024-09-13T16:50:30.116691+0000","last_active":"2024-09-13T16:50:35.087857+0000","last_peered":"2024-09-13T16:50:35.087857+0000","last_clean":"2024-09-13T16:50:35.087857+0000","last_became_active":"2024-09-13T16:50:30.115262+0000","last_became_peered":"2024-09-13T16:50:30.115262+0000","last_unstale":"2024-09-13T16:50:35.087857+0000","last_undegraded":"2024-09-13T16:50:35.087857+0000","last_fullsized":"2024-09-13T16:50:35.087857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.506161+0000","last_change":"2024-09-13T16:50:35.499612+0000","last_active":"2024-09-13T16:50:35.506161+0000","last_peered":"2024-09-13T16:50:35.506161+0000","last_clean":"2024-09-13T16:50:35.506161+0000","last_became_active":"2024-09-13T16:50:35.498882+0000","last_became_peered":"2024-09-13T16:50:35.498882+0000","last_unstale":"2024-09-13T16:50:35.506161+0000","last_undegraded":"2024-09-13T16:50:35.506161+0000","last_fullsized":"2024-09-13T16:50:35.506161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:14.360209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092696+0000","last_change":"2024-09-13T16:50:30.601337+0000","last_active":"2024-09-13T16:50:35.092696+0000","last_peered":"2024-09-13T16:50:35.092696+0000","last_clean":"2024-09-13T16:50:35.092696+0000","last_became_active":"2024-09-13T16:50:30.601073+0000","last_became_peered":"2024-09-13T16:50:30.601073+0000","last_unstale":"2024-09-13T16:50:35.092696+0000","last_undegraded":"2024-09-13T16:50:35.092696+0000","last_fullsized":"2024-09-13T16:50:35.092696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.117003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402285+0000","last_change":"2024-09-13T16:50:30.109801+0000","last_active":"2024-09-13T16:50:35.402285+0000","last_peered":"2024-09-13T16:50:35.402285+0000","last_clean":"2024-09-13T16:50:35.402285+0000","last_became_active":"2024-09-13T16:50:30.107936+0000","last_became_peered":"2024-09-13T16:50:30.107936+0000","last_unstale":"2024-09-13T16:50:35.402285+0000","last_undegraded":"2024-09-13T16:50:35.402285+0000","last_fullsized":"2024-09-13T16:50:35.402285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462892+0000","last_change":"2024-09-13T16:50:34.640936+0000","last_active":"2024-09-13T16:50:35.462892+0000","last_peered":"2024-09-13T16:50:35.462892+0000","last_clean":"2024-09-13T16:50:35.462892+0000","last_became_active":"2024-09-13T16:50:34.640606+0000","last_became_peered":"2024-09-13T16:50:34.640606+0000","last_unstale":"2024-09-13T16:50:35.462892+0000","last_undegraded":"2024-09-13T16:50:35.462892+0000","last_fullsized":"2024-09-13T16:50:35.462892+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500787+0000","last_change":"2024-09-13T16:50:30.620140+0000","last_active":"2024-09-13T16:50:35.500787+0000","last_peered":"2024-09-13T16:50:35.500787+0000","last_clean":"2024-09-13T16:50:35.500787+0000","last_became_active":"2024-09-13T16:50:30.619960+0000","last_became_peered":"2024-09-13T16:50:30.619960+0000","last_unstale":"2024-09-13T16:50:35.500787+0000","last_undegraded":"2024-09-13T16:50:35.500787+0000","last_fullsized":"2024-09-13T16:50:35.500787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:28.715573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.088162+0000","last_change":"2024-09-13T16:50:30.104959+0000","last_active":"2024-09-13T16:50:35.088162+0000","last_peered":"2024-09-13T16:50:35.088162+0000","last_clean":"2024-09-13T16:50:35.088162+0000","last_became_active":"2024-09-13T16:50:30.104709+0000","last_became_peered":"2024-09-13T16:50:30.104709+0000","last_unstale":"2024-09-13T16:50:35.088162+0000","last_undegraded":"2024-09-13T16:50:35.088162+0000","last_fullsized":"2024-09-13T16:50:35.088162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.828286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.505477+0000","last_change":"2024-09-13T16:50:35.501120+0000","last_active":"2024-09-13T16:50:35.505477+0000","last_peered":"2024-09-13T16:50:35.505477+0000","last_clean":"2024-09-13T16:50:35.505477+0000","last_became_active":"2024-09-13T16:50:35.500500+0000","last_became_peered":"2024-09-13T16:50:35.500500+0000","last_unstale":"2024-09-13T16:50:35.505477+0000","last_undegraded":"2024-09-13T16:50:35.505477+0000","last_fullsized":"2024-09-13T16:50:35.505477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:58.021389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499025+0000","last_change":"2024-09-13T16:50:30.615966+0000","last_active":"2024-09-13T16:50:35.499025+0000","last_peered":"2024-09-13T16:50:35.499025+0000","last_clean":"2024-09-13T16:50:35.499025+0000","last_became_active":"2024-09-13T16:50:30.615750+0000","last_became_peered":"2024-09-13T16:50:30.615750+0000","last_unstale":"2024-09-13T16:50:35.499025+0000","last_undegraded":"2024-09-13T16:50:35.499025+0000","last_fullsized":"2024-09-13T16:50:35.499025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501690+0000","last_change":"2024-09-13T16:50:30.112955+0000","last_active":"2024-09-13T16:50:35.501690+0000","last_peered":"2024-09-13T16:50:35.501690+0000","last_clean":"2024-09-13T16:50:35.501690+0000","last_became_active":"2024-09-13T16:50:30.111633+0000","last_became_peered":"2024-09-13T16:50:30.111633+0000","last_unstale":"2024-09-13T16:50:35.501690+0000","last_undegraded":"2024-09-13T16:50:35.501690+0000","last_fullsized":"2024-09-13T16:50:35.501690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.499166+0000","last_change":"2024-09-13T16:50:34.636875+0000","last_active":"2024-09-13T16:50:35.499166+0000","last_peered":"2024-09-13T16:50:35.499166+0000","last_clean":"2024-09-13T16:50:35.499166+0000","last_became_active":"2024-09-13T16:50:34.636350+0000","last_became_peered":"2024-09-13T16:50:34.636350+0000","last_unstale":"2024-09-13T16:50:35.499166+0000","last_undegraded":"2024-09-13T16:50:35.499166+0000","last_fullsized":"2024-09-13T16:50:35.499166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:32.980957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498929+0000","last_change":"2024-09-13T16:50:30.617707+0000","last_active":"2024-09-13T16:50:35.498929+0000","last_peered":"2024-09-13T16:50:35.498929+0000","last_clean":"2024-09-13T16:50:35.498929+0000","last_became_active":"2024-09-13T16:50:30.617279+0000","last_became_peered":"2024-09-13T16:50:30.617279+0000","last_unstale":"2024-09-13T16:50:35.498929+0000","last_undegraded":"2024-09-13T16:50:35.498929+0000","last_fullsized":"2024-09-13T16:50:35.498929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087386+0000","last_change":"2024-09-13T16:50:30.107834+0000","last_active":"2024-09-13T16:50:35.087386+0000","last_peered":"2024-09-13T16:50:35.087386+0000","last_clean":"2024-09-13T16:50:35.087386+0000","last_became_active":"2024-09-13T16:50:30.107457+0000","last_became_peered":"2024-09-13T16:50:30.107457+0000","last_unstale":"2024-09-13T16:50:35.087386+0000","last_undegraded":"2024-09-13T16:50:35.087386+0000","last_fullsized":"2024-09-13T16:50:35.087386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463011+0000","last_change":"2024-09-13T16:50:34.635554+0000","last_active":"2024-09-13T16:50:35.463011+0000","last_peered":"2024-09-13T16:50:35.463011+0000","last_clean":"2024-09-13T16:50:35.463011+0000","last_became_active":"2024-09-13T16:50:34.635194+0000","last_became_peered":"2024-09-13T16:50:34.635194+0000","last_unstale":"2024-09-13T16:50:35.463011+0000","last_undegraded":"2024-09-13T16:50:35.463011+0000","last_fullsized":"2024-09-13T16:50:35.463011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:54.702371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091123+0000","last_change":"2024-09-13T16:50:30.612793+0000","last_active":"2024-09-13T16:50:35.091123+0000","last_peered":"2024-09-13T16:50:35.091123+0000","last_clean":"2024-09-13T16:50:35.091123+0000","last_became_active":"2024-09-13T16:50:30.612243+0000","last_became_peered":"2024-09-13T16:50:30.612243+0000","last_unstale":"2024-09-13T16:50:35.091123+0000","last_undegraded":"2024-09-13T16:50:35.091123+0000","last_fullsized":"2024-09-13T16:50:35.091123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091094+0000","last_change":"2024-09-13T16:50:30.097963+0000","last_active":"2024-09-13T16:50:35.091094+0000","last_peered":"2024-09-13T16:50:35.091094+0000","last_clean":"2024-09-13T16:50:35.091094+0000","last_became_active":"2024-09-13T16:50:30.097015+0000","last_became_peered":"2024-09-13T16:50:30.097015+0000","last_unstale":"2024-09-13T16:50:35.091094+0000","last_undegraded":"2024-09-13T16:50:35.091094+0000","last_fullsized":"2024-09-13T16:50:35.091094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:40.126246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.514416+0000","last_change":"2024-09-13T16:50:35.512427+0000","last_active":"2024-09-13T16:50:35.514416+0000","last_peered":"2024-09-13T16:50:35.514416+0000","last_clean":"2024-09-13T16:50:35.514416+0000","last_became_active":"2024-09-13T16:50:35.512074+0000","last_became_peered":"2024-09-13T16:50:35.512074+0000","last_unstale":"2024-09-13T16:50:35.514416+0000","last_undegraded":"2024-09-13T16:50:35.514416+0000","last_fullsized":"2024-09-13T16:50:35.514416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:37.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498993+0000","last_change":"2024-09-13T16:50:30.617270+0000","last_active":"2024-09-13T16:50:35.498993+0000","last_peered":"2024-09-13T16:50:35.498993+0000","last_clean":"2024-09-13T16:50:35.498993+0000","last_became_active":"2024-09-13T16:50:30.617047+0000","last_became_peered":"2024-09-13T16:50:30.617047+0000","last_unstale":"2024-09-13T16:50:35.498993+0000","last_undegraded":"2024-09-13T16:50:35.498993+0000","last_fullsized":"2024-09-13T16:50:35.498993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091892+0000","last_change":"2024-09-13T16:50:30.109703+0000","last_active":"2024-09-13T16:50:35.091892+0000","last_peered":"2024-09-13T16:50:35.091892+0000","last_clean":"2024-09-13T16:50:35.091892+0000","last_became_active":"2024-09-13T16:50:30.109427+0000","last_became_peered":"2024-09-13T16:50:30.109427+0000","last_unstale":"2024-09-13T16:50:35.091892+0000","last_undegraded":"2024-09-13T16:50:35.091892+0000","last_fullsized":"2024-09-13T16:50:35.091892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509958+0000","last_change":"2024-09-13T16:50:35.501616+0000","last_active":"2024-09-13T16:50:35.509958+0000","last_peered":"2024-09-13T16:50:35.509958+0000","last_clean":"2024-09-13T16:50:35.509958+0000","last_became_active":"2024-09-13T16:50:35.500854+0000","last_became_peered":"2024-09-13T16:50:35.500854+0000","last_unstale":"2024-09-13T16:50:35.509958+0000","last_undegraded":"2024-09-13T16:50:35.509958+0000","last_fullsized":"2024-09-13T16:50:35.509958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:18.549426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462681+0000","last_change":"2024-09-13T16:50:30.619507+0000","last_active":"2024-09-13T16:50:35.462681+0000","last_peered":"2024-09-13T16:50:35.462681+0000","last_clean":"2024-09-13T16:50:35.462681+0000","last_became_active":"2024-09-13T16:50:30.619122+0000","last_became_peered":"2024-09-13T16:50:30.619122+0000","last_unstale":"2024-09-13T16:50:35.462681+0000","last_undegraded":"2024-09-13T16:50:35.462681+0000","last_fullsized":"2024-09-13T16:50:35.462681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:00.203031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463139+0000","last_change":"2024-09-13T16:50:30.099206+0000","last_active":"2024-09-13T16:50:35.463139+0000","last_peered":"2024-09-13T16:50:35.463139+0000","last_clean":"2024-09-13T16:50:35.463139+0000","last_became_active":"2024-09-13T16:50:30.098688+0000","last_became_peered":"2024-09-13T16:50:30.098688+0000","last_unstale":"2024-09-13T16:50:35.463139+0000","last_undegraded":"2024-09-13T16:50:35.463139+0000","last_fullsized":"2024-09-13T16:50:35.463139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.228773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.505292+0000","last_change":"2024-09-13T16:50:35.502406+0000","last_active":"2024-09-13T16:50:35.505292+0000","last_peered":"2024-09-13T16:50:35.505292+0000","last_clean":"2024-09-13T16:50:35.505292+0000","last_became_active":"2024-09-13T16:50:35.502014+0000","last_became_peered":"2024-09-13T16:50:35.502014+0000","last_unstale":"2024-09-13T16:50:35.505292+0000","last_undegraded":"2024-09-13T16:50:35.505292+0000","last_fullsized":"2024-09-13T16:50:35.505292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:58.615917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498570+0000","last_change":"2024-09-13T16:50:30.613053+0000","last_active":"2024-09-13T16:50:35.498570+0000","last_peered":"2024-09-13T16:50:35.498570+0000","last_clean":"2024-09-13T16:50:35.498570+0000","last_became_active":"2024-09-13T16:50:30.612775+0000","last_became_peered":"2024-09-13T16:50:30.612775+0000","last_unstale":"2024-09-13T16:50:35.498570+0000","last_undegraded":"2024-09-13T16:50:35.498570+0000","last_fullsized":"2024-09-13T16:50:35.498570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:54.244738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T16:50:47.108380+0000","last_change":"2024-09-13T16:50:30.111095+0000","last_active":"2024-09-13T16:50:47.108380+0000","last_peered":"2024-09-13T16:50:47.108380+0000","last_clean":"2024-09-13T16:50:47.108380+0000","last_became_active":"2024-09-13T16:50:30.110886+0000","last_became_peered":"2024-09-13T16:50:30.110886+0000","last_unstale":"2024-09-13T16:50:47.108380+0000","last_undegraded":"2024-09-13T16:50:47.108380+0000","last_fullsized":"2024-09-13T16:50:47.108380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401424+0000","last_change":"2024-09-13T16:50:34.641283+0000","last_active":"2024-09-13T16:50:35.401424+0000","last_peered":"2024-09-13T16:50:35.401424+0000","last_clean":"2024-09-13T16:50:35.401424+0000","last_became_active":"2024-09-13T16:50:34.636369+0000","last_became_peered":"2024-09-13T16:50:34.636369+0000","last_unstale":"2024-09-13T16:50:35.401424+0000","last_undegraded":"2024-09-13T16:50:35.401424+0000","last_fullsized":"2024-09-13T16:50:35.401424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:46.586048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.502226+0000","last_change":"2024-09-13T16:50:30.611541+0000","last_active":"2024-09-13T16:50:35.502226+0000","last_peered":"2024-09-13T16:50:35.502226+0000","last_clean":"2024-09-13T16:50:35.502226+0000","last_became_active":"2024-09-13T16:50:30.611046+0000","last_became_peered":"2024-09-13T16:50:30.611046+0000","last_unstale":"2024-09-13T16:50:35.502226+0000","last_undegraded":"2024-09-13T16:50:35.502226+0000","last_fullsized":"2024-09-13T16:50:35.502226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:51.423821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402184+0000","last_change":"2024-09-13T16:50:30.111117+0000","last_active":"2024-09-13T16:50:35.402184+0000","last_peered":"2024-09-13T16:50:35.402184+0000","last_clean":"2024-09-13T16:50:35.402184+0000","last_became_active":"2024-09-13T16:50:30.110857+0000","last_became_peered":"2024-09-13T16:50:30.110857+0000","last_unstale":"2024-09-13T16:50:35.402184+0000","last_undegraded":"2024-09-13T16:50:35.402184+0000","last_fullsized":"2024-09-13T16:50:35.402184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:21.938851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500960+0000","last_change":"2024-09-13T16:50:34.634925+0000","last_active":"2024-09-13T16:50:35.500960+0000","last_peered":"2024-09-13T16:50:35.500960+0000","last_clean":"2024-09-13T16:50:35.500960+0000","last_became_active":"2024-09-13T16:50:34.634329+0000","last_became_peered":"2024-09-13T16:50:34.634329+0000","last_unstale":"2024-09-13T16:50:35.500960+0000","last_undegraded":"2024-09-13T16:50:35.500960+0000","last_fullsized":"2024-09-13T16:50:35.500960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:30.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463375+0000","last_change":"2024-09-13T16:50:30.609227+0000","last_active":"2024-09-13T16:50:35.463375+0000","last_peered":"2024-09-13T16:50:35.463375+0000","last_clean":"2024-09-13T16:50:35.463375+0000","last_became_active":"2024-09-13T16:50:30.608863+0000","last_became_peered":"2024-09-13T16:50:30.608863+0000","last_unstale":"2024-09-13T16:50:35.463375+0000","last_undegraded":"2024-09-13T16:50:35.463375+0000","last_fullsized":"2024-09-13T16:50:35.463375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093039+0000","last_change":"2024-09-13T16:50:30.109198+0000","last_active":"2024-09-13T16:50:35.093039+0000","last_peered":"2024-09-13T16:50:35.093039+0000","last_clean":"2024-09-13T16:50:35.093039+0000","last_became_active":"2024-09-13T16:50:30.108896+0000","last_became_peered":"2024-09-13T16:50:30.108896+0000","last_unstale":"2024-09-13T16:50:35.093039+0000","last_undegraded":"2024-09-13T16:50:35.093039+0000","last_fullsized":"2024-09-13T16:50:35.093039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:29:58.933647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456883+0000","last_change":"2024-09-13T16:50:34.630350+0000","last_active":"2024-09-13T16:50:35.456883+0000","last_peered":"2024-09-13T16:50:35.456883+0000","last_clean":"2024-09-13T16:50:35.456883+0000","last_became_active":"2024-09-13T16:50:34.626215+0000","last_became_peered":"2024-09-13T16:50:34.626215+0000","last_unstale":"2024-09-13T16:50:35.456883+0000","last_undegraded":"2024-09-13T16:50:35.456883+0000","last_fullsized":"2024-09-13T16:50:35.456883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:51.705201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092550+0000","last_change":"2024-09-13T16:50:30.603528+0000","last_active":"2024-09-13T16:50:35.092550+0000","last_peered":"2024-09-13T16:50:35.092550+0000","last_clean":"2024-09-13T16:50:35.092550+0000","last_became_active":"2024-09-13T16:50:30.603265+0000","last_became_peered":"2024-09-13T16:50:30.603265+0000","last_unstale":"2024-09-13T16:50:35.092550+0000","last_undegraded":"2024-09-13T16:50:35.092550+0000","last_fullsized":"2024-09-13T16:50:35.092550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:45.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093205+0000","last_change":"2024-09-13T16:50:30.112126+0000","last_active":"2024-09-13T16:50:35.093205+0000","last_peered":"2024-09-13T16:50:35.093205+0000","last_clean":"2024-09-13T16:50:35.093205+0000","last_became_active":"2024-09-13T16:50:30.111844+0000","last_became_peered":"2024-09-13T16:50:30.111844+0000","last_unstale":"2024-09-13T16:50:35.093205+0000","last_undegraded":"2024-09-13T16:50:35.093205+0000","last_fullsized":"2024-09-13T16:50:35.093205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:50.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.511284+0000","last_change":"2024-09-13T16:50:35.507739+0000","last_active":"2024-09-13T16:50:35.511284+0000","last_peered":"2024-09-13T16:50:35.511284+0000","last_clean":"2024-09-13T16:50:35.511284+0000","last_became_active":"2024-09-13T16:50:35.505030+0000","last_became_peered":"2024-09-13T16:50:35.505030+0000","last_unstale":"2024-09-13T16:50:35.511284+0000","last_undegraded":"2024-09-13T16:50:35.511284+0000","last_fullsized":"2024-09-13T16:50:35.511284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:29.316344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.509712+0000","last_change":"2024-09-13T16:50:35.506793+0000","last_active":"2024-09-13T16:50:35.509712+0000","last_peered":"2024-09-13T16:50:35.509712+0000","last_clean":"2024-09-13T16:50:35.509712+0000","last_became_active":"2024-09-13T16:50:35.505380+0000","last_became_peered":"2024-09-13T16:50:35.505380+0000","last_unstale":"2024-09-13T16:50:35.509712+0000","last_undegraded":"2024-09-13T16:50:35.509712+0000","last_fullsized":"2024-09-13T16:50:35.509712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:58.754567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.400877+0000","last_change":"2024-09-13T16:50:30.601660+0000","last_active":"2024-09-13T16:50:35.400877+0000","last_peered":"2024-09-13T16:50:35.400877+0000","last_clean":"2024-09-13T16:50:35.400877+0000","last_became_active":"2024-09-13T16:50:30.601315+0000","last_became_peered":"2024-09-13T16:50:30.601315+0000","last_unstale":"2024-09-13T16:50:35.400877+0000","last_undegraded":"2024-09-13T16:50:35.400877+0000","last_fullsized":"2024-09-13T16:50:35.400877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091503+0000","last_change":"2024-09-13T16:50:30.098656+0000","last_active":"2024-09-13T16:50:35.091503+0000","last_peered":"2024-09-13T16:50:35.091503+0000","last_clean":"2024-09-13T16:50:35.091503+0000","last_became_active":"2024-09-13T16:50:30.098137+0000","last_became_peered":"2024-09-13T16:50:30.098137+0000","last_unstale":"2024-09-13T16:50:35.091503+0000","last_undegraded":"2024-09-13T16:50:35.091503+0000","last_fullsized":"2024-09-13T16:50:35.091503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:42.952299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093095+0000","last_change":"2024-09-13T16:50:34.632360+0000","last_active":"2024-09-13T16:50:35.093095+0000","last_peered":"2024-09-13T16:50:35.093095+0000","last_clean":"2024-09-13T16:50:35.093095+0000","last_became_active":"2024-09-13T16:50:34.631958+0000","last_became_peered":"2024-09-13T16:50:34.631958+0000","last_unstale":"2024-09-13T16:50:35.093095+0000","last_undegraded":"2024-09-13T16:50:35.093095+0000","last_fullsized":"2024-09-13T16:50:35.093095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:30.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456997+0000","last_change":"2024-09-13T16:50:30.609936+0000","last_active":"2024-09-13T16:50:35.456997+0000","last_peered":"2024-09-13T16:50:35.456997+0000","last_clean":"2024-09-13T16:50:35.456997+0000","last_became_active":"2024-09-13T16:50:30.609683+0000","last_became_peered":"2024-09-13T16:50:30.609683+0000","last_unstale":"2024-09-13T16:50:35.456997+0000","last_undegraded":"2024-09-13T16:50:35.456997+0000","last_fullsized":"2024-09-13T16:50:35.456997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:32.908616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456931+0000","last_change":"2024-09-13T16:50:30.103999+0000","last_active":"2024-09-13T16:50:35.456931+0000","last_peered":"2024-09-13T16:50:35.456931+0000","last_clean":"2024-09-13T16:50:35.456931+0000","last_became_active":"2024-09-13T16:50:30.103531+0000","last_became_peered":"2024-09-13T16:50:30.103531+0000","last_unstale":"2024-09-13T16:50:35.456931+0000","last_undegraded":"2024-09-13T16:50:35.456931+0000","last_fullsized":"2024-09-13T16:50:35.456931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.828286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091972+0000","last_change":"2024-09-13T16:50:30.103767+0000","last_active":"2024-09-13T16:50:35.091972+0000","last_peered":"2024-09-13T16:50:35.091972+0000","last_clean":"2024-09-13T16:50:35.091972+0000","last_became_active":"2024-09-13T16:50:30.103519+0000","last_became_peered":"2024-09-13T16:50:30.103519+0000","last_unstale":"2024-09-13T16:50:35.091972+0000","last_undegraded":"2024-09-13T16:50:35.091972+0000","last_fullsized":"2024-09-13T16:50:35.091972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.073431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463060+0000","last_change":"2024-09-13T16:50:30.618557+0000","last_active":"2024-09-13T16:50:35.463060+0000","last_peered":"2024-09-13T16:50:35.463060+0000","last_clean":"2024-09-13T16:50:35.463060+0000","last_became_active":"2024-09-13T16:50:30.618308+0000","last_became_peered":"2024-09-13T16:50:30.618308+0000","last_unstale":"2024-09-13T16:50:35.463060+0000","last_undegraded":"2024-09-13T16:50:35.463060+0000","last_fullsized":"2024-09-13T16:50:35.463060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:45.160388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087606+0000","last_change":"2024-09-13T16:50:34.639885+0000","last_active":"2024-09-13T16:50:35.087606+0000","last_peered":"2024-09-13T16:50:35.087606+0000","last_clean":"2024-09-13T16:50:35.087606+0000","last_became_active":"2024-09-13T16:50:34.639399+0000","last_became_peered":"2024-09-13T16:50:34.639399+0000","last_unstale":"2024-09-13T16:50:35.087606+0000","last_undegraded":"2024-09-13T16:50:35.087606+0000","last_fullsized":"2024-09-13T16:50:35.087606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:56:36.425638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456657+0000","last_change":"2024-09-13T16:50:30.102739+0000","last_active":"2024-09-13T16:50:35.456657+0000","last_peered":"2024-09-13T16:50:35.456657+0000","last_clean":"2024-09-13T16:50:35.456657+0000","last_became_active":"2024-09-13T16:50:30.102448+0000","last_became_peered":"2024-09-13T16:50:30.102448+0000","last_unstale":"2024-09-13T16:50:35.456657+0000","last_undegraded":"2024-09-13T16:50:35.456657+0000","last_fullsized":"2024-09-13T16:50:35.456657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:29.053925+0000","last_clean_scrub_stamp":"2024-09-13T16:50:29.053925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:38.882189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.501017+0000","last_change":"2024-09-13T16:50:30.597447+0000","last_active":"2024-09-13T16:50:35.501017+0000","last_peered":"2024-09-13T16:50:35.501017+0000","last_clean":"2024-09-13T16:50:35.501017+0000","last_became_active":"2024-09-13T16:50:30.592891+0000","last_became_peered":"2024-09-13T16:50:30.592891+0000","last_unstale":"2024-09-13T16:50:35.501017+0000","last_undegraded":"2024-09-13T16:50:35.501017+0000","last_fullsized":"2024-09-13T16:50:35.501017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:13.892340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456015+0000","last_change":"2024-09-13T16:50:34.630131+0000","last_active":"2024-09-13T16:50:35.456015+0000","last_peered":"2024-09-13T16:50:35.456015+0000","last_clean":"2024-09-13T16:50:35.456015+0000","last_became_active":"2024-09-13T16:50:34.629601+0000","last_became_peered":"2024-09-13T16:50:34.629601+0000","last_unstale":"2024-09-13T16:50:35.456015+0000","last_undegraded":"2024-09-13T16:50:35.456015+0000","last_fullsized":"2024-09-13T16:50:35.456015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:33.582408+0000","last_clean_scrub_stamp":"2024-09-13T16:50:33.582408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:38.058431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.090962+0000","last_change":"2024-09-13T16:50:30.612982+0000","last_active":"2024-09-13T16:50:35.090962+0000","last_peered":"2024-09-13T16:50:35.090962+0000","last_clean":"2024-09-13T16:50:35.090962+0000","last_became_active":"2024-09-13T16:50:30.612577+0000","last_became_peered":"2024-09-13T16:50:30.612577+0000","last_unstale":"2024-09-13T16:50:35.090962+0000","last_undegraded":"2024-09-13T16:50:35.090962+0000","last_fullsized":"2024-09-13T16:50:35.090962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.087762+0000","last_change":"2024-09-13T16:50:30.594530+0000","last_active":"2024-09-13T16:50:35.087762+0000","last_peered":"2024-09-13T16:50:35.087762+0000","last_clean":"2024-09-13T16:50:35.087762+0000","last_became_active":"2024-09-13T16:50:30.594248+0000","last_became_peered":"2024-09-13T16:50:30.594248+0000","last_unstale":"2024-09-13T16:50:35.087762+0000","last_undegraded":"2024-09-13T16:50:35.087762+0000","last_fullsized":"2024-09-13T16:50:35.087762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:22.950313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091814+0000","last_change":"2024-09-13T16:50:30.602412+0000","last_active":"2024-09-13T16:50:35.091814+0000","last_peered":"2024-09-13T16:50:35.091814+0000","last_clean":"2024-09-13T16:50:35.091814+0000","last_became_active":"2024-09-13T16:50:30.602146+0000","last_became_peered":"2024-09-13T16:50:30.602146+0000","last_unstale":"2024-09-13T16:50:35.091814+0000","last_undegraded":"2024-09-13T16:50:35.091814+0000","last_fullsized":"2024-09-13T16:50:35.091814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:35.586398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462908+0000","last_change":"2024-09-13T16:50:30.606003+0000","last_active":"2024-09-13T16:50:35.462908+0000","last_peered":"2024-09-13T16:50:35.462908+0000","last_clean":"2024-09-13T16:50:35.462908+0000","last_became_active":"2024-09-13T16:50:30.605562+0000","last_became_peered":"2024-09-13T16:50:30.605562+0000","last_unstale":"2024-09-13T16:50:35.462908+0000","last_undegraded":"2024-09-13T16:50:35.462908+0000","last_fullsized":"2024-09-13T16:50:35.462908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456431+0000","last_change":"2024-09-13T16:50:30.617701+0000","last_active":"2024-09-13T16:50:35.456431+0000","last_peered":"2024-09-13T16:50:35.456431+0000","last_clean":"2024-09-13T16:50:35.456431+0000","last_became_active":"2024-09-13T16:50:30.616581+0000","last_became_peered":"2024-09-13T16:50:30.616581+0000","last_unstale":"2024-09-13T16:50:35.456431+0000","last_undegraded":"2024-09-13T16:50:35.456431+0000","last_fullsized":"2024-09-13T16:50:35.456431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:52.041529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.401854+0000","last_change":"2024-09-13T16:50:30.599260+0000","last_active":"2024-09-13T16:50:35.401854+0000","last_peered":"2024-09-13T16:50:35.401854+0000","last_clean":"2024-09-13T16:50:35.401854+0000","last_became_active":"2024-09-13T16:50:30.599018+0000","last_became_peered":"2024-09-13T16:50:30.599018+0000","last_unstale":"2024-09-13T16:50:35.401854+0000","last_undegraded":"2024-09-13T16:50:35.401854+0000","last_fullsized":"2024-09-13T16:50:35.401854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093088+0000","last_change":"2024-09-13T16:50:30.613429+0000","last_active":"2024-09-13T16:50:35.093088+0000","last_peered":"2024-09-13T16:50:35.093088+0000","last_clean":"2024-09-13T16:50:35.093088+0000","last_became_active":"2024-09-13T16:50:30.613163+0000","last_became_peered":"2024-09-13T16:50:30.613163+0000","last_unstale":"2024-09-13T16:50:35.093088+0000","last_undegraded":"2024-09-13T16:50:35.093088+0000","last_fullsized":"2024-09-13T16:50:35.093088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456736+0000","last_change":"2024-09-13T16:50:30.607903+0000","last_active":"2024-09-13T16:50:35.456736+0000","last_peered":"2024-09-13T16:50:35.456736+0000","last_clean":"2024-09-13T16:50:35.456736+0000","last_became_active":"2024-09-13T16:50:30.607618+0000","last_became_peered":"2024-09-13T16:50:30.607618+0000","last_unstale":"2024-09-13T16:50:35.456736+0000","last_undegraded":"2024-09-13T16:50:35.456736+0000","last_fullsized":"2024-09-13T16:50:35.456736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:01.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091481+0000","last_change":"2024-09-13T16:50:30.603058+0000","last_active":"2024-09-13T16:50:35.091481+0000","last_peered":"2024-09-13T16:50:35.091481+0000","last_clean":"2024-09-13T16:50:35.091481+0000","last_became_active":"2024-09-13T16:50:30.602556+0000","last_became_peered":"2024-09-13T16:50:30.602556+0000","last_unstale":"2024-09-13T16:50:35.091481+0000","last_undegraded":"2024-09-13T16:50:35.091481+0000","last_fullsized":"2024-09-13T16:50:35.091481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456543+0000","last_change":"2024-09-13T16:50:30.599518+0000","last_active":"2024-09-13T16:50:35.456543+0000","last_peered":"2024-09-13T16:50:35.456543+0000","last_clean":"2024-09-13T16:50:35.456543+0000","last_became_active":"2024-09-13T16:50:30.599242+0000","last_became_peered":"2024-09-13T16:50:30.599242+0000","last_unstale":"2024-09-13T16:50:35.456543+0000","last_undegraded":"2024-09-13T16:50:35.456543+0000","last_fullsized":"2024-09-13T16:50:35.456543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091768+0000","last_change":"2024-09-13T16:50:30.610239+0000","last_active":"2024-09-13T16:50:35.091768+0000","last_peered":"2024-09-13T16:50:35.091768+0000","last_clean":"2024-09-13T16:50:35.091768+0000","last_became_active":"2024-09-13T16:50:30.609978+0000","last_became_peered":"2024-09-13T16:50:30.609978+0000","last_unstale":"2024-09-13T16:50:35.091768+0000","last_undegraded":"2024-09-13T16:50:35.091768+0000","last_fullsized":"2024-09-13T16:50:35.091768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:28.715573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462797+0000","last_change":"2024-09-13T16:50:30.618976+0000","last_active":"2024-09-13T16:50:35.462797+0000","last_peered":"2024-09-13T16:50:35.462797+0000","last_clean":"2024-09-13T16:50:35.462797+0000","last_became_active":"2024-09-13T16:50:30.618644+0000","last_became_peered":"2024-09-13T16:50:30.618644+0000","last_unstale":"2024-09-13T16:50:35.462797+0000","last_undegraded":"2024-09-13T16:50:35.462797+0000","last_fullsized":"2024-09-13T16:50:35.462797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:52.041529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092551+0000","last_change":"2024-09-13T16:50:30.609748+0000","last_active":"2024-09-13T16:50:35.092551+0000","last_peered":"2024-09-13T16:50:35.092551+0000","last_clean":"2024-09-13T16:50:35.092551+0000","last_became_active":"2024-09-13T16:50:30.609504+0000","last_became_peered":"2024-09-13T16:50:30.609504+0000","last_unstale":"2024-09-13T16:50:35.092551+0000","last_undegraded":"2024-09-13T16:50:35.092551+0000","last_fullsized":"2024-09-13T16:50:35.092551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500181+0000","last_change":"2024-09-13T16:50:30.618417+0000","last_active":"2024-09-13T16:50:35.500181+0000","last_peered":"2024-09-13T16:50:35.500181+0000","last_clean":"2024-09-13T16:50:35.500181+0000","last_became_active":"2024-09-13T16:50:30.618114+0000","last_became_peered":"2024-09-13T16:50:30.618114+0000","last_unstale":"2024-09-13T16:50:35.500181+0000","last_undegraded":"2024-09-13T16:50:35.500181+0000","last_fullsized":"2024-09-13T16:50:35.500181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:25.758134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.088098+0000","last_change":"2024-09-13T16:50:30.604041+0000","last_active":"2024-09-13T16:50:35.088098+0000","last_peered":"2024-09-13T16:50:35.088098+0000","last_clean":"2024-09-13T16:50:35.088098+0000","last_became_active":"2024-09-13T16:50:30.599336+0000","last_became_peered":"2024-09-13T16:50:30.599336+0000","last_unstale":"2024-09-13T16:50:35.088098+0000","last_undegraded":"2024-09-13T16:50:35.088098+0000","last_fullsized":"2024-09-13T16:50:35.088098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:13.892340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093370+0000","last_change":"2024-09-13T16:50:30.610847+0000","last_active":"2024-09-13T16:50:35.093370+0000","last_peered":"2024-09-13T16:50:35.093370+0000","last_clean":"2024-09-13T16:50:35.093370+0000","last_became_active":"2024-09-13T16:50:30.610581+0000","last_became_peered":"2024-09-13T16:50:30.610581+0000","last_unstale":"2024-09-13T16:50:35.093370+0000","last_undegraded":"2024-09-13T16:50:35.093370+0000","last_fullsized":"2024-09-13T16:50:35.093370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:21.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091388+0000","last_change":"2024-09-13T16:50:30.618144+0000","last_active":"2024-09-13T16:50:35.091388+0000","last_peered":"2024-09-13T16:50:35.091388+0000","last_clean":"2024-09-13T16:50:35.091388+0000","last_became_active":"2024-09-13T16:50:30.617861+0000","last_became_peered":"2024-09-13T16:50:30.617861+0000","last_unstale":"2024-09-13T16:50:35.091388+0000","last_undegraded":"2024-09-13T16:50:35.091388+0000","last_fullsized":"2024-09-13T16:50:35.091388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.400884+0000","last_change":"2024-09-13T16:50:30.590573+0000","last_active":"2024-09-13T16:50:35.400884+0000","last_peered":"2024-09-13T16:50:35.400884+0000","last_clean":"2024-09-13T16:50:35.400884+0000","last_became_active":"2024-09-13T16:50:30.590338+0000","last_became_peered":"2024-09-13T16:50:30.590338+0000","last_unstale":"2024-09-13T16:50:35.400884+0000","last_undegraded":"2024-09-13T16:50:35.400884+0000","last_fullsized":"2024-09-13T16:50:35.400884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:30.798565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.462638+0000","last_change":"2024-09-13T16:50:30.615260+0000","last_active":"2024-09-13T16:50:35.462638+0000","last_peered":"2024-09-13T16:50:35.462638+0000","last_clean":"2024-09-13T16:50:35.462638+0000","last_became_active":"2024-09-13T16:50:30.614529+0000","last_became_peered":"2024-09-13T16:50:30.614529+0000","last_unstale":"2024-09-13T16:50:35.462638+0000","last_undegraded":"2024-09-13T16:50:35.462638+0000","last_fullsized":"2024-09-13T16:50:35.462638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:08.657623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.455914+0000","last_change":"2024-09-13T16:50:30.603994+0000","last_active":"2024-09-13T16:50:35.455914+0000","last_peered":"2024-09-13T16:50:35.455914+0000","last_clean":"2024-09-13T16:50:35.455914+0000","last_became_active":"2024-09-13T16:50:30.603610+0000","last_became_peered":"2024-09-13T16:50:30.603610+0000","last_unstale":"2024-09-13T16:50:35.455914+0000","last_undegraded":"2024-09-13T16:50:35.455914+0000","last_fullsized":"2024-09-13T16:50:35.455914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:56.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.502132+0000","last_change":"2024-09-13T16:50:30.603873+0000","last_active":"2024-09-13T16:50:35.502132+0000","last_peered":"2024-09-13T16:50:35.502132+0000","last_clean":"2024-09-13T16:50:35.502132+0000","last_became_active":"2024-09-13T16:50:30.603571+0000","last_became_peered":"2024-09-13T16:50:30.603571+0000","last_unstale":"2024-09-13T16:50:35.502132+0000","last_undegraded":"2024-09-13T16:50:35.502132+0000","last_fullsized":"2024-09-13T16:50:35.502132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:22.950313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092808+0000","last_change":"2024-09-13T16:50:30.607038+0000","last_active":"2024-09-13T16:50:35.092808+0000","last_peered":"2024-09-13T16:50:35.092808+0000","last_clean":"2024-09-13T16:50:35.092808+0000","last_became_active":"2024-09-13T16:50:30.606788+0000","last_became_peered":"2024-09-13T16:50:30.606788+0000","last_unstale":"2024-09-13T16:50:35.092808+0000","last_undegraded":"2024-09-13T16:50:35.092808+0000","last_fullsized":"2024-09-13T16:50:35.092808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:28.367692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.093030+0000","last_change":"2024-09-13T16:50:30.599427+0000","last_active":"2024-09-13T16:50:35.093030+0000","last_peered":"2024-09-13T16:50:35.093030+0000","last_clean":"2024-09-13T16:50:35.093030+0000","last_became_active":"2024-09-13T16:50:30.599149+0000","last_became_peered":"2024-09-13T16:50:30.599149+0000","last_unstale":"2024-09-13T16:50:35.093030+0000","last_undegraded":"2024-09-13T16:50:35.093030+0000","last_fullsized":"2024-09-13T16:50:35.093030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:27.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500732+0000","last_change":"2024-09-13T16:50:30.615710+0000","last_active":"2024-09-13T16:50:35.500732+0000","last_peered":"2024-09-13T16:50:35.500732+0000","last_clean":"2024-09-13T16:50:35.500732+0000","last_became_active":"2024-09-13T16:50:30.615444+0000","last_became_peered":"2024-09-13T16:50:30.615444+0000","last_unstale":"2024-09-13T16:50:35.500732+0000","last_undegraded":"2024-09-13T16:50:35.500732+0000","last_fullsized":"2024-09-13T16:50:35.500732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:35.586398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.456590+0000","last_change":"2024-09-13T16:50:30.599700+0000","last_active":"2024-09-13T16:50:35.456590+0000","last_peered":"2024-09-13T16:50:35.456590+0000","last_clean":"2024-09-13T16:50:35.456590+0000","last_became_active":"2024-09-13T16:50:30.599132+0000","last_became_peered":"2024-09-13T16:50:30.599132+0000","last_unstale":"2024-09-13T16:50:35.456590+0000","last_undegraded":"2024-09-13T16:50:35.456590+0000","last_fullsized":"2024-09-13T16:50:35.456590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498763+0000","last_change":"2024-09-13T16:50:30.617544+0000","last_active":"2024-09-13T16:50:35.498763+0000","last_peered":"2024-09-13T16:50:35.498763+0000","last_clean":"2024-09-13T16:50:35.498763+0000","last_became_active":"2024-09-13T16:50:30.617008+0000","last_became_peered":"2024-09-13T16:50:30.617008+0000","last_unstale":"2024-09-13T16:50:35.498763+0000","last_undegraded":"2024-09-13T16:50:35.498763+0000","last_fullsized":"2024-09-13T16:50:35.498763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.091720+0000","last_change":"2024-09-13T16:50:30.613899+0000","last_active":"2024-09-13T16:50:35.091720+0000","last_peered":"2024-09-13T16:50:35.091720+0000","last_clean":"2024-09-13T16:50:35.091720+0000","last_became_active":"2024-09-13T16:50:30.613520+0000","last_became_peered":"2024-09-13T16:50:30.613520+0000","last_unstale":"2024-09-13T16:50:35.091720+0000","last_undegraded":"2024-09-13T16:50:35.091720+0000","last_fullsized":"2024-09-13T16:50:35.091720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:49.574392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.498780+0000","last_change":"2024-09-13T16:50:30.617169+0000","last_active":"2024-09-13T16:50:35.498780+0000","last_peered":"2024-09-13T16:50:35.498780+0000","last_clean":"2024-09-13T16:50:35.498780+0000","last_became_active":"2024-09-13T16:50:30.616608+0000","last_became_peered":"2024-09-13T16:50:30.616608+0000","last_unstale":"2024-09-13T16:50:35.498780+0000","last_undegraded":"2024-09-13T16:50:35.498780+0000","last_fullsized":"2024-09-13T16:50:35.498780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.117003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.092229+0000","last_change":"2024-09-13T16:50:30.601494+0000","last_active":"2024-09-13T16:50:35.092229+0000","last_peered":"2024-09-13T16:50:35.092229+0000","last_clean":"2024-09-13T16:50:35.092229+0000","last_became_active":"2024-09-13T16:50:30.601235+0000","last_became_peered":"2024-09-13T16:50:30.601235+0000","last_unstale":"2024-09-13T16:50:35.092229+0000","last_undegraded":"2024-09-13T16:50:35.092229+0000","last_fullsized":"2024-09-13T16:50:35.092229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:54.244738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.463054+0000","last_change":"2024-09-13T16:50:30.611789+0000","last_active":"2024-09-13T16:50:35.463054+0000","last_peered":"2024-09-13T16:50:35.463054+0000","last_clean":"2024-09-13T16:50:35.463054+0000","last_became_active":"2024-09-13T16:50:30.608646+0000","last_became_peered":"2024-09-13T16:50:30.608646+0000","last_unstale":"2024-09-13T16:50:35.463054+0000","last_undegraded":"2024-09-13T16:50:35.463054+0000","last_fullsized":"2024-09-13T16:50:35.463054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.145790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.402335+0000","last_change":"2024-09-13T16:50:30.609892+0000","last_active":"2024-09-13T16:50:35.402335+0000","last_peered":"2024-09-13T16:50:35.402335+0000","last_clean":"2024-09-13T16:50:35.402335+0000","last_became_active":"2024-09-13T16:50:30.609664+0000","last_became_peered":"2024-09-13T16:50:30.609664+0000","last_unstale":"2024-09-13T16:50:35.402335+0000","last_undegraded":"2024-09-13T16:50:35.402335+0000","last_fullsized":"2024-09-13T16:50:35.402335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:13.557730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:50:35.500628+0000","last_change":"2024-09-13T16:50:30.607977+0000","last_active":"2024-09-13T16:50:35.500628+0000","last_peered":"2024-09-13T16:50:35.500628+0000","last_clean":"2024-09-13T16:50:35.500628+0000","last_became_active":"2024-09-13T16:50:30.607735+0000","last_became_peered":"2024-09-13T16:50:30.607735+0000","last_unstale":"2024-09-13T16:50:35.500628+0000","last_undegraded":"2024-09-13T16:50:35.500628+0000","last_fullsized":"2024-09-13T16:50:35.500628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:30.065387+0000","last_clean_scrub_stamp":"2024-09-13T16:50:30.065387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:21.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":864256,"data_stored":697795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.631}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5780000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":278528,"data_stored":106971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":106845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":697851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:51:04.289 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T16:51:04.289 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T16:51:04.289 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T16:51:04.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T16:51:04.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.497+0000 7f3a83200700 1 Processor -- start 2024-09-13T16:51:04.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.497+0000 7f3a83200700 1 -- start start 2024-09-13T16:51:04.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a83200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a840711b0 0x7f3a840b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a83200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a840b4130 con 0x7f3a840b1bd0 2024-09-13T16:51:04.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a83200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a840ce920 con 0x7f3a840b24f0 2024-09-13T16:51:04.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a83200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a840cdb00 con 0x7f3a840711b0 2024-09-13T16:51:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3a840b1bd0 0x7f3a840b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34614/0 (socket says 172.21.15.3:34614) 2024-09-13T16:51:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81e00700 1 -- 172.21.15.3:0/1577920500 learned_addr learned my addr 172.21.15.3:0/1577920500 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a79400700 1 --2- 172.21.15.3:0/1577920500 >> v2:172.21.15.157:3300/0 conn(0x7f3a840711b0 0x7f3a840b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292373475 0 0) 0x7f3a74001490 con 0x7f3a840b1bd0 2024-09-13T16:51:04.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a680035d0 con 0x7f3a840b1bd0 2024-09-13T16:51:04.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560413481 0 0) 0x7f3a840ce920 con 0x7f3a840b24f0 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a74001490 con 0x7f3a840b24f0 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 901230769 0 0) 0x7f3a680035d0 con 0x7f3a840b1bd0 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 >> v2:172.21.15.157:3300/0 conn(0x7f3a840711b0 msgr2=0x7f3a840b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 --2- 172.21.15.3:0/1577920500 >> v2:172.21.15.157:3300/0 conn(0x7f3a840711b0 0x7f3a840b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 >> v1:172.21.15.157:6789/0 conn(0x7f3a840b24f0 legacy=0x7f3a841634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840cdd10 con 0x7f3a840b1bd0 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f3a74001b80 con 0x7f3a840b1bd0 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 >> v1:172.21.15.3:6789/0 conn(0x7f3a840b1bd0 legacy=0x7f3a840b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68003bb0 0x7f3a68003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a68004870 0x7f3a68004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a68005230 0x7f3a6800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840cdb00 con 0x7f3a68005230 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800c160 con 0x7f3a68003bb0 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800c320 con 0x7f3a68004870 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a79400700 1 --2- 172.21.15.3:0/1577920500 >> v2:172.21.15.157:3300/0 conn(0x7f3a840711b0 0x7f3a840b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.501+0000 7f3a82800700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a68005230 0x7f3a6800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a81e00700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68003bb0 0x7f3a68003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a79400700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a68004870 0x7f3a68004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a82800700 1 -- 172.21.15.3:0/1577920500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a68004870 msgr2=0x7f3a68004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a82800700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a68004870 0x7f3a68004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a82800700 1 -- 172.21.15.3:0/1577920500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68003bb0 msgr2=0x7f3a68003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a82800700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68003bb0 0x7f3a68003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a82800700 1 -- 172.21.15.3:0/1577920500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680035d0 con 0x7f3a68005230 2024-09-13T16:51:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a82800700 1 -- 172.21.15.3:0/1577920500 reap_dead start 2024-09-13T16:51:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a82800700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a68005230 0x7f3a6800ba30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c001950 tx=0x7f3a680058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c02bbcc9ee49f0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c001d80 con 0x7f3a68005230 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0052d0 con 0x7f3a68005230 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a81400700 1 -- 172.21.15.3:0/1577920500 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c001440 con 0x7f3a68005230 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a83200700 1 -- 172.21.15.3:0/1577920500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a68005230 msgr2=0x7f3a6800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a83200700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a68005230 0x7f3a6800ba30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c001950 tx=0x7f3a680058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a83200700 1 -- 172.21.15.3:0/1577920500 shutdown_connections 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a83200700 1 --2- 172.21.15.3:0/1577920500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a68005230 0x7f3a6800ba30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a83200700 1 -- 172.21.15.3:0/1577920500 >> 172.21.15.3:0/1577920500 conn(0x7f3a840b0cf0 msgr2=0x7f3a840b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a83200700 1 -- 172.21.15.3:0/1577920500 shutdown_connections 2024-09-13T16:51:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.505+0000 7f3a83200700 1 -- 172.21.15.3:0/1577920500 wait complete. 2024-09-13T16:51:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a83200700 1 Processor -- start 2024-09-13T16:51:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a83200700 1 -- start start 2024-09-13T16:51:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a83200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68005230 0x7f3a8413a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a83200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8414c2b0 0x7f3a8414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a83200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8413cec0 0x7f3a8413d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a83200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8414ecb0 con 0x7f3a8414c2b0 2024-09-13T16:51:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a83200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8413a540 con 0x7f3a68005230 2024-09-13T16:51:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a83200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8414e410 con 0x7f3a8413cec0 2024-09-13T16:51:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8414c2b0 0x7f3a8414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8414c2b0 0x7f3a8414dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32878/0 (socket says 172.21.15.3:32878) 2024-09-13T16:51:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 -- 172.21.15.3:0/1146395407 learned_addr learned my addr 172.21.15.3:0/1146395407 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a82800700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8413cec0 0x7f3a8413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a81e00700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68005230 0x7f3a8413a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 -- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8413cec0 msgr2=0x7f3a8413d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8413cec0 0x7f3a8413d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 -- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68005230 msgr2=0x7f3a8413a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68005230 0x7f3a8413a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 -- 172.21.15.3:0/1146395407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8414e6f0 con 0x7f3a8414c2b0 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a82800700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8413cec0 0x7f3a8413d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a81e00700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68005230 0x7f3a8413a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.509+0000 7f3a79400700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8414c2b0 0x7f3a8414dbe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c011cd0 tx=0x7f3a6c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.513+0000 7f3a7aa00700 1 -- 172.21.15.3:0/1146395407 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00fb30 con 0x7f3a8414c2b0 2024-09-13T16:51:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.513+0000 7f3a7aa00700 1 -- 172.21.15.3:0/1146395407 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c00fc90 con 0x7f3a8414c2b0 2024-09-13T16:51:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.513+0000 7f3a7aa00700 1 -- 172.21.15.3:0/1146395407 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c014440 con 0x7f3a8414c2b0 2024-09-13T16:51:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.513+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8414e9d0 con 0x7f3a8414c2b0 2024-09-13T16:51:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.513+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8414c710 con 0x7f3a8414c2b0 2024-09-13T16:51:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.513+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a840cb840 con 0x7f3a8414c2b0 2024-09-13T16:51:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.513+0000 7f3a7aa00700 1 -- 172.21.15.3:0/1146395407 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a6c0145a0 con 0x7f3a8414c2b0 2024-09-13T16:51:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.517+0000 7f3a7aa00700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a60066730 0x7f3a60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.517+0000 7f3a7aa00700 1 -- 172.21.15.3:0/1146395407 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3a6c084d40 con 0x7f3a8414c2b0 2024-09-13T16:51:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.517+0000 7f3a81e00700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a60066730 0x7f3a60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.517+0000 7f3a81e00700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a60066730 0x7f3a60068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400a4f0 tx=0x7f3a74006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.525+0000 7f3a7aa00700 1 -- 172.21.15.3:0/1146395407 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3a6c051d20 con 0x7f3a8414c2b0 2024-09-13T16:51:04.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.745+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3a840ae600 con 0x7f3a8414c2b0 2024-09-13T16:51:04.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.745+0000 7f3a7aa00700 1 -- 172.21.15.3:0/1146395407 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3a6c022080 con 0x7f3a8414c2b0 2024-09-13T16:51:04.750 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:04.750 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T16:51:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a60066730 msgr2=0x7f3a60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a60066730 0x7f3a60068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400a4f0 tx=0x7f3a74006040 comp rx=0 tx=0).stop 2024-09-13T16:51:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8414c2b0 msgr2=0x7f3a8414dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8414c2b0 0x7f3a8414dbe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c011cd0 tx=0x7f3a6c00f520 comp rx=0 tx=0).stop 2024-09-13T16:51:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 shutdown_connections 2024-09-13T16:51:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a60066730 0x7f3a60068bf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8413cec0 0x7f3a8413d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8414c2b0 0x7f3a8414dbe0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 --2- 172.21.15.3:0/1146395407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a68005230 0x7f3a8413a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 >> 172.21.15.3:0/1146395407 conn(0x7f3a840b0cf0 msgr2=0x7f3a841547f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 shutdown_connections 2024-09-13T16:51:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.749+0000 7f3a83200700 1 -- 172.21.15.3:0/1146395407 wait complete. 2024-09-13T16:51:04.775 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T16:51:04.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:04.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.953+0000 7ff30d400700 1 Processor -- start 2024-09-13T16:51:04.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.953+0000 7ff30d400700 1 -- start start 2024-09-13T16:51:04.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.957+0000 7ff30d400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806a390 0x7ff3080b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.957+0000 7ff30d400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3080ae8a0 con 0x7ff30806b3d0 2024-09-13T16:51:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.957+0000 7ff30d400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3080ae2e0 con 0x7ff30806aff0 2024-09-13T16:51:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.957+0000 7ff30d400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30815b810 con 0x7ff30806a390 2024-09-13T16:51:04.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.957+0000 7ff306800700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff30806aff0 0x7ff3080b0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:48952/0 (socket says 172.21.15.3:48952) 2024-09-13T16:51:04.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.957+0000 7ff307200700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff30806b3d0 0x7ff3080b15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34622/0 (socket says 172.21.15.3:34622) 2024-09-13T16:51:04.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.957+0000 7ff306800700 1 -- 172.21.15.3:0/1328031242 learned_addr learned my addr 172.21.15.3:0/1328031242 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:04.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.957+0000 7ff305e00700 1 --2- 172.21.15.3:0/1328031242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806a390 0x7ff3080b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305400700 1 -- 172.21.15.3:0/1328031242 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399668476 0 0) 0x7ff3080ae8a0 con 0x7ff30806b3d0 2024-09-13T16:51:04.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305400700 1 -- 172.21.15.3:0/1328031242 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2ec0035d0 con 0x7ff30806b3d0 2024-09-13T16:51:04.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305400700 1 -- 172.21.15.3:0/1328031242 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791980842 0 0) 0x7ff3080ae2e0 con 0x7ff30806aff0 2024-09-13T16:51:04.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305e00700 1 -- 172.21.15.3:0/1328031242 >> v1:172.21.15.157:6789/0 conn(0x7ff30806aff0 legacy=0x7ff3080b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305e00700 1 -- 172.21.15.3:0/1328031242 >> v1:172.21.15.3:6789/0 conn(0x7ff30806b3d0 legacy=0x7ff3080b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305e00700 1 -- 172.21.15.3:0/1328031242 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080ae5c0 con 0x7ff30806a390 2024-09-13T16:51:04.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305e00700 1 --2- 172.21.15.3:0/1328031242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806a390 0x7ff3080b4cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009880 tx=0x7ff2f4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a96e8a8a0adb64e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305400700 1 -- 172.21.15.3:0/1328031242 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011830 con 0x7ff30806a390 2024-09-13T16:51:04.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305400700 1 -- 172.21.15.3:0/1328031242 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f4011990 con 0x7ff30806a390 2024-09-13T16:51:04.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff305400700 1 -- 172.21.15.3:0/1328031242 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011d20 con 0x7ff30806a390 2024-09-13T16:51:04.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff30d400700 1 -- 172.21.15.3:0/1328031242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806a390 msgr2=0x7ff3080b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff30d400700 1 --2- 172.21.15.3:0/1328031242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806a390 0x7ff3080b4cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009880 tx=0x7ff2f4009b90 comp rx=0 tx=0).stop 2024-09-13T16:51:04.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff30d400700 1 -- 172.21.15.3:0/1328031242 shutdown_connections 2024-09-13T16:51:04.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff30d400700 1 --2- 172.21.15.3:0/1328031242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806a390 0x7ff3080b4cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff30d400700 1 -- 172.21.15.3:0/1328031242 >> 172.21.15.3:0/1328031242 conn(0x7ff3080b3f20 msgr2=0x7ff3080b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:04.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff30d400700 1 -- 172.21.15.3:0/1328031242 shutdown_connections 2024-09-13T16:51:04.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.961+0000 7ff30d400700 1 -- 172.21.15.3:0/1328031242 wait complete. 2024-09-13T16:51:04.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff30d400700 1 Processor -- start 2024-09-13T16:51:04.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff30d400700 1 -- start start 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff30d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30806a390 0x7ff308146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff30d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30806aff0 0x7ff308164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff30d400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806b3d0 0x7ff308165310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff30d400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff308154230 con 0x7ff30806aff0 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff30d400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30814f1e0 con 0x7ff30806a390 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff30d400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30814f4c0 con 0x7ff30806b3d0 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30806aff0 0x7ff308164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30806aff0 0x7ff308164dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32894/0 (socket says 172.21.15.3:32894) 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 -- 172.21.15.3:0/1971402098 learned_addr learned my addr 172.21.15.3:0/1971402098 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff307200700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806b3d0 0x7ff308165310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff306800700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30806a390 0x7ff308146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 -- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806b3d0 msgr2=0x7ff308165310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806b3d0 0x7ff308165310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 -- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30806a390 msgr2=0x7ff308146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30806a390 0x7ff308146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 -- 172.21.15.3:0/1971402098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f40094b0 con 0x7ff30806aff0 2024-09-13T16:51:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.965+0000 7ff305e00700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30806aff0 0x7ff308164dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4013040 tx=0x7ff2f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.969+0000 7ff2fea00700 1 -- 172.21.15.3:0/1971402098 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4019a80 con 0x7ff30806aff0 2024-09-13T16:51:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.969+0000 7ff2fea00700 1 -- 172.21.15.3:0/1971402098 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f401e070 con 0x7ff30806aff0 2024-09-13T16:51:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.969+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30814f7a0 con 0x7ff30806aff0 2024-09-13T16:51:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.969+0000 7ff2fea00700 1 -- 172.21.15.3:0/1971402098 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4019e00 con 0x7ff30806aff0 2024-09-13T16:51:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.969+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3081479e0 con 0x7ff30806aff0 2024-09-13T16:51:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.969+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30814fcc0 con 0x7ff30806aff0 2024-09-13T16:51:04.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.969+0000 7ff2fea00700 1 -- 172.21.15.3:0/1971402098 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff2f402d450 con 0x7ff30806aff0 2024-09-13T16:51:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.973+0000 7ff2fea00700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff2e8066730 0x7ff2e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.973+0000 7ff2fea00700 1 -- 172.21.15.3:0/1971402098 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff2f4085610 con 0x7ff30806aff0 2024-09-13T16:51:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.973+0000 7ff306800700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff2e8066730 0x7ff2e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.973+0000 7ff306800700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff2e8066730 0x7ff2e8068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8000f60 tx=0x7ff2f8007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:04.981+0000 7ff2fea00700 1 -- 172.21.15.3:0/1971402098 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff30814fcc0 con 0x7ff30806aff0 2024-09-13T16:51:05.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.173+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff30814fcc0 con 0x7ff30806aff0 2024-09-13T16:51:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.173+0000 7ff2fea00700 1 -- 172.21.15.3:0/1971402098 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7ff30814fcc0 con 0x7ff30806aff0 2024-09-13T16:51:05.180 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:05.180 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff2e8066730 msgr2=0x7ff2e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff2e8066730 0x7ff2e8068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8000f60 tx=0x7ff2f8007c00 comp rx=0 tx=0).stop 2024-09-13T16:51:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30806aff0 msgr2=0x7ff308164dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30806aff0 0x7ff308164dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4013040 tx=0x7ff2f4004770 comp rx=0 tx=0).stop 2024-09-13T16:51:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 shutdown_connections 2024-09-13T16:51:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff2e8066730 0x7ff2e8068bf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30806b3d0 0x7ff308165310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30806aff0 0x7ff308164dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 --2- 172.21.15.3:0/1971402098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30806a390 0x7ff308146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 >> 172.21.15.3:0/1971402098 conn(0x7ff3080b3f20 msgr2=0x7ff30815af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 shutdown_connections 2024-09-13T16:51:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.177+0000 7ff30d400700 1 -- 172.21.15.3:0/1971402098 wait complete. 2024-09-13T16:51:05.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:05.207 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.401+0000 7f1b17e00700 1 Processor -- start 2024-09-13T16:51:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.401+0000 7f1b17e00700 1 -- start start 2024-09-13T16:51:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b17e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b181444f0 0x7f1b180d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b17e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b180b5a30 con 0x7f1b180d0680 2024-09-13T16:51:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b17e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b180b5d10 con 0x7f1b1815ed90 2024-09-13T16:51:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b17e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b180cd5b0 con 0x7f1b181444f0 2024-09-13T16:51:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b16000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1b180d0680 0x7f1b1815e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34624/0 (socket says 172.21.15.3:34624) 2024-09-13T16:51:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b16000700 1 -- 172.21.15.3:0/2474583162 learned_addr learned my addr 172.21.15.3:0/2474583162 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b16a00700 1 --2- 172.21.15.3:0/2474583162 >> v2:172.21.15.157:3300/0 conn(0x7f1b181444f0 0x7f1b180d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b15600700 1 -- 172.21.15.3:0/2474583162 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132855425 0 0) 0x7f1b180b5a30 con 0x7f1b180d0680 2024-09-13T16:51:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b15600700 1 -- 172.21.15.3:0/2474583162 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1afc0035d0 con 0x7f1b180d0680 2024-09-13T16:51:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b16a00700 1 -- 172.21.15.3:0/2474583162 >> v1:172.21.15.157:6789/0 conn(0x7f1b1815ed90 legacy=0x7f1b180cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b15600700 1 -- 172.21.15.3:0/2474583162 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3472765301 0 0) 0x7f1afc0035d0 con 0x7f1b180d0680 2024-09-13T16:51:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b16a00700 1 -- 172.21.15.3:0/2474583162 >> v1:172.21.15.3:6789/0 conn(0x7f1b180d0680 legacy=0x7f1b1815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.405+0000 7f1b16a00700 1 -- 172.21.15.3:0/2474583162 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180cd890 con 0x7f1b181444f0 2024-09-13T16:51:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b16a00700 1 --2- 172.21.15.3:0/2474583162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b181444f0 0x7f1b180d0140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009850 tx=0x7f1b08009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e82b386f3a6bd71 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b15600700 1 -- 172.21.15.3:0/2474583162 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04001670 con 0x7f1b181444f0 2024-09-13T16:51:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b15600700 1 -- 172.21.15.3:0/2474583162 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08010590 con 0x7f1b181444f0 2024-09-13T16:51:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b15600700 1 -- 172.21.15.3:0/2474583162 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08010940 con 0x7f1b181444f0 2024-09-13T16:51:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 -- 172.21.15.3:0/2474583162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b181444f0 msgr2=0x7f1b180d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 --2- 172.21.15.3:0/2474583162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b181444f0 0x7f1b180d0140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009850 tx=0x7f1b08009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 -- 172.21.15.3:0/2474583162 shutdown_connections 2024-09-13T16:51:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 --2- 172.21.15.3:0/2474583162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b181444f0 0x7f1b180d0140 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 -- 172.21.15.3:0/2474583162 >> 172.21.15.3:0/2474583162 conn(0x7f1b180721a0 msgr2=0x7f1b1806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 -- 172.21.15.3:0/2474583162 shutdown_connections 2024-09-13T16:51:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 -- 172.21.15.3:0/2474583162 wait complete. 2024-09-13T16:51:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 Processor -- start 2024-09-13T16:51:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.409+0000 7f1b17e00700 1 -- start start 2024-09-13T16:51:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b180d0680 0x7f1b181571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b181444f0 0x7f1b1813c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b1815ed90 0x7f1b1813ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1813e310 con 0x7f1b1815ed90 2024-09-13T16:51:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1813d260 con 0x7f1b180d0680 2024-09-13T16:51:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1813d9e0 con 0x7f1b181444f0 2024-09-13T16:51:05.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b1815ed90 0x7f1b1813ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b1815ed90 0x7f1b1813ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32906/0 (socket says 172.21.15.3:32906) 2024-09-13T16:51:05.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 -- 172.21.15.3:0/1858348358 learned_addr learned my addr 172.21.15.3:0/1858348358 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:05.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b16a00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b180d0680 0x7f1b181571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 -- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b181444f0 msgr2=0x7f1b1813c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b16000700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b181444f0 0x7f1b1813c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b181444f0 0x7f1b1813c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 -- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b180d0680 msgr2=0x7f1b181571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b180d0680 0x7f1b181571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 -- 172.21.15.3:0/1858348358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080094b0 con 0x7f1b1815ed90 2024-09-13T16:51:05.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b16000700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b181444f0 0x7f1b1813c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:05.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17400700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b1815ed90 0x7f1b1813ca30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000cc10 tx=0x7f1b1000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b0ea00700 1 -- 172.21.15.3:0/1858348358 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10003e50 con 0x7f1b1815ed90 2024-09-13T16:51:05.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b0ea00700 1 -- 172.21.15.3:0/1858348358 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1000a7e0 con 0x7f1b1815ed90 2024-09-13T16:51:05.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1813dcc0 con 0x7f1b1815ed90 2024-09-13T16:51:05.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b0ea00700 1 -- 172.21.15.3:0/1858348358 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000ab90 con 0x7f1b1815ed90 2024-09-13T16:51:05.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.413+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1813cf70 con 0x7f1b1815ed90 2024-09-13T16:51:05.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.417+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1813e4c0 con 0x7f1b1815ed90 2024-09-13T16:51:05.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.417+0000 7f1b0ea00700 1 -- 172.21.15.3:0/1858348358 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b1000acf0 con 0x7f1b1815ed90 2024-09-13T16:51:05.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.421+0000 7f1b0ea00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1af8066760 0x7f1af8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.421+0000 7f1b0ea00700 1 -- 172.21.15.3:0/1858348358 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1b10084550 con 0x7f1b1815ed90 2024-09-13T16:51:05.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.421+0000 7f1b16a00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1af8066760 0x7f1af8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.421+0000 7f1b16a00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1af8066760 0x7f1af8068c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b18142e80 tx=0x7f1b0801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.425+0000 7f1b0ea00700 1 -- 172.21.15.3:0/1858348358 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b1813e4c0 con 0x7f1b1815ed90 2024-09-13T16:51:05.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.609+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b1813e450 con 0x7f1b1815ed90 2024-09-13T16:51:05.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.609+0000 7f1b0ea00700 1 -- 172.21.15.3:0/1858348358 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b100515c0 con 0x7f1b1815ed90 2024-09-13T16:51:05.615 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:05.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1af8066760 msgr2=0x7f1af8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1af8066760 0x7f1af8068c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b18142e80 tx=0x7f1b0801c040 comp rx=0 tx=0).stop 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b1815ed90 msgr2=0x7f1b1813ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b1815ed90 0x7f1b1813ca30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000cc10 tx=0x7f1b1000a060 comp rx=0 tx=0).stop 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 shutdown_connections 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1af8066760 0x7f1af8068c20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b1815ed90 0x7f1b1813ca30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b181444f0 0x7f1b1813c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 --2- 172.21.15.3:0/1858348358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b180d0680 0x7f1b181571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 >> 172.21.15.3:0/1858348358 conn(0x7f1b180721a0 msgr2=0x7f1b18066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 shutdown_connections 2024-09-13T16:51:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.613+0000 7f1b17e00700 1 -- 172.21.15.3:0/1858348358 wait complete. 2024-09-13T16:51:05.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:05.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.809+0000 7f22abe00700 1 Processor -- start 2024-09-13T16:51:05.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.809+0000 7f22abe00700 1 -- start start 2024-09-13T16:51:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22abe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f22ac15cd30 0x7f22ac0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22abe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22ac15d110 0x7f22ac0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22abe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac06a390 con 0x7f22ac15d560 2024-09-13T16:51:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22abe00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac06e0c0 con 0x7f22ac15cd30 2024-09-13T16:51:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22abe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac15daf0 con 0x7f22ac15d110 2024-09-13T16:51:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aaa00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f22ac15cd30 0x7f22ac0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aaa00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f22ac15cd30 0x7f22ac0b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32912/0 (socket says 172.21.15.3:32912) 2024-09-13T16:51:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aaa00700 1 -- 172.21.15.3:0/3291079231 learned_addr learned my addr 172.21.15.3:0/3291079231 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aa000700 1 --2- 172.21.15.3:0/3291079231 >> v2:172.21.15.157:3300/0 conn(0x7f22ac15d110 0x7f22ac0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aaa00700 1 -- 172.21.15.3:0/3291079231 >> v2:172.21.15.157:3300/0 conn(0x7f22ac15d110 msgr2=0x7f22ac0719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aaa00700 1 --2- 172.21.15.3:0/3291079231 >> v2:172.21.15.157:3300/0 conn(0x7f22ac15d110 0x7f22ac0719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aaa00700 1 -- 172.21.15.3:0/3291079231 >> v1:172.21.15.157:6789/0 conn(0x7f22ac15d560 legacy=0x7f22ac160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aaa00700 1 -- 172.21.15.3:0/3291079231 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac15ddd0 con 0x7f22ac15cd30 2024-09-13T16:51:05.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22aaa00700 1 --2- 172.21.15.3:0/3291079231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15cd30 0x7f22ac0b2ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f22ac1619a0 tx=0x7f229c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c20b5b41e71ab3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22a9600700 1 -- 172.21.15.3:0/3291079231 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00b8a0 con 0x7f22ac15cd30 2024-09-13T16:51:05.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22a9600700 1 -- 172.21.15.3:0/3291079231 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c00ba00 con 0x7f22ac15cd30 2024-09-13T16:51:05.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22a9600700 1 -- 172.21.15.3:0/3291079231 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00bd90 con 0x7f22ac15cd30 2024-09-13T16:51:05.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22abe00700 1 -- 172.21.15.3:0/3291079231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15cd30 msgr2=0x7f22ac0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.813+0000 7f22abe00700 1 --2- 172.21.15.3:0/3291079231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15cd30 0x7f22ac0b2ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f22ac1619a0 tx=0x7f229c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 -- 172.21.15.3:0/3291079231 shutdown_connections 2024-09-13T16:51:05.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 --2- 172.21.15.3:0/3291079231 >> v2:172.21.15.157:3300/0 conn(0x7f22ac15d110 0x7f22ac0719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 --2- 172.21.15.3:0/3291079231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15cd30 0x7f22ac0b2ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 -- 172.21.15.3:0/3291079231 >> 172.21.15.3:0/3291079231 conn(0x7f22ac0b23d0 msgr2=0x7f22ac0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:05.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 -- 172.21.15.3:0/3291079231 shutdown_connections 2024-09-13T16:51:05.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 -- 172.21.15.3:0/3291079231 wait complete. 2024-09-13T16:51:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 Processor -- start 2024-09-13T16:51:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 -- start start 2024-09-13T16:51:05.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22ac15cd30 0x7f22ac09a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22ac15d110 0x7f22ac0ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15d560 0x7f22ac0ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac09b420 con 0x7f22ac15d560 2024-09-13T16:51:05.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac09b6c0 con 0x7f22ac15d110 2024-09-13T16:51:05.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.817+0000 7f22abe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac09b9a0 con 0x7f22ac15cd30 2024-09-13T16:51:05.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22ac15cd30 0x7f22ac09a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aa000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22ac15d110 0x7f22ac0ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22ac15cd30 0x7f22ac09a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:36554/0 (socket says 172.21.15.3:36554) 2024-09-13T16:51:05.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 -- 172.21.15.3:0/27762131 learned_addr learned my addr 172.21.15.3:0/27762131 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:05.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22ab400700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15d560 0x7f22ac0ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 -- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22ac15d110 msgr2=0x7f22ac0ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22ac15d110 0x7f22ac0ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 -- 172.21.15.3:0/27762131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15d560 msgr2=0x7f22ac0ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15d560 0x7f22ac0ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 -- 172.21.15.3:0/27762131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c009570 con 0x7f22ac15cd30 2024-09-13T16:51:05.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22aaa00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22ac15cd30 0x7f22ac09a1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f22ac06e8e0 tx=0x7f229c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22a3400700 1 -- 172.21.15.3:0/27762131 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0036c0 con 0x7f22ac15cd30 2024-09-13T16:51:05.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22a3400700 1 -- 172.21.15.3:0/27762131 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c01f070 con 0x7f22ac15cd30 2024-09-13T16:51:05.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22a3400700 1 -- 172.21.15.3:0/27762131 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c003a80 con 0x7f22ac15cd30 2024-09-13T16:51:05.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac09bc80 con 0x7f22ac15cd30 2024-09-13T16:51:05.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.821+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac0aeeb0 con 0x7f22ac15cd30 2024-09-13T16:51:05.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.825+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac0b2ef0 con 0x7f22ac15cd30 2024-09-13T16:51:05.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.825+0000 7f22a3400700 1 -- 172.21.15.3:0/27762131 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f229c021030 con 0x7f22ac15cd30 2024-09-13T16:51:05.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.825+0000 7f22a3400700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f228c066780 0x7f228c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.825+0000 7f22a3400700 1 -- 172.21.15.3:0/27762131 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f229c0859a0 con 0x7f22ac15cd30 2024-09-13T16:51:05.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.825+0000 7f22aa000700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f228c066780 0x7f228c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.829+0000 7f22aa000700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f228c066780 0x7f228c068c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0b1720 tx=0x7f2298008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:05.837+0000 7f22a3400700 1 -- 172.21.15.3:0/27762131 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f229c052b40 con 0x7f22ac15cd30 2024-09-13T16:51:06.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.021+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22ac09bae0 con 0x7f22ac15cd30 2024-09-13T16:51:06.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.021+0000 7f22a3400700 1 -- 172.21.15.3:0/27762131 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22ac09bae0 con 0x7f22ac15cd30 2024-09-13T16:51:06.025 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f228c066780 msgr2=0x7f228c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f228c066780 0x7f228c068c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0b1720 tx=0x7f2298008040 comp rx=0 tx=0).stop 2024-09-13T16:51:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22ac15cd30 msgr2=0x7f22ac09a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22ac15cd30 0x7f22ac09a1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f22ac06e8e0 tx=0x7f229c004770 comp rx=0 tx=0).stop 2024-09-13T16:51:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 shutdown_connections 2024-09-13T16:51:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f228c066780 0x7f228c068c40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac15d560 0x7f22ac0ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22ac15d110 0x7f22ac0ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 --2- 172.21.15.3:0/27762131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22ac15cd30 0x7f22ac09a1e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 >> 172.21.15.3:0/27762131 conn(0x7f22ac0b23d0 msgr2=0x7f22ac159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 shutdown_connections 2024-09-13T16:51:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.025+0000 7f22abe00700 1 -- 172.21.15.3:0/27762131 wait complete. 2024-09-13T16:51:06.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:06.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.237+0000 7f45ffe00700 1 Processor -- start 2024-09-13T16:51:06.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.237+0000 7f45ffe00700 1 -- start start 2024-09-13T16:51:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45ffe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f460015c990 0x7f46000b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45ffe00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46000b2230 con 0x7f46000cd390 2024-09-13T16:51:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45ffe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46000b1190 con 0x7f46000b0cc0 2024-09-13T16:51:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45ffe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46000b1470 con 0x7f460015c990 2024-09-13T16:51:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fe000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f46000cd390 0x7f46000b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34632/0 (socket says 172.21.15.3:34632) 2024-09-13T16:51:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fe000700 1 -- 172.21.15.3:0/857604069 learned_addr learned my addr 172.21.15.3:0/857604069 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fea00700 1 --2- 172.21.15.3:0/857604069 >> v2:172.21.15.157:3300/0 conn(0x7f460015c990 0x7f46000b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fd600700 1 -- 172.21.15.3:0/857604069 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212931903 0 0) 0x7f46000b2230 con 0x7f46000cd390 2024-09-13T16:51:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fd600700 1 -- 172.21.15.3:0/857604069 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e40035d0 con 0x7f46000cd390 2024-09-13T16:51:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fea00700 1 -- 172.21.15.3:0/857604069 >> v1:172.21.15.157:6789/0 conn(0x7f46000b0cc0 legacy=0x7f460006b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fea00700 1 -- 172.21.15.3:0/857604069 >> v1:172.21.15.3:6789/0 conn(0x7f46000cd390 legacy=0x7f46000b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fea00700 1 -- 172.21.15.3:0/857604069 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46000b1750 con 0x7f460015c990 2024-09-13T16:51:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fea00700 1 --2- 172.21.15.3:0/857604069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460015c990 0x7f46000b0780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009850 tx=0x7f45f0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725ce4bb97e7ce40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fd600700 1 -- 172.21.15.3:0/857604069 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0011830 con 0x7f460015c990 2024-09-13T16:51:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fd600700 1 -- 172.21.15.3:0/857604069 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f0011990 con 0x7f460015c990 2024-09-13T16:51:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.241+0000 7f45fd600700 1 -- 172.21.15.3:0/857604069 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0011d40 con 0x7f460015c990 2024-09-13T16:51:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- 172.21.15.3:0/857604069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460015c990 msgr2=0x7f46000b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 --2- 172.21.15.3:0/857604069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460015c990 0x7f46000b0780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009850 tx=0x7f45f0009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:06.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- 172.21.15.3:0/857604069 shutdown_connections 2024-09-13T16:51:06.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 --2- 172.21.15.3:0/857604069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460015c990 0x7f46000b0780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- 172.21.15.3:0/857604069 >> 172.21.15.3:0/857604069 conn(0x7f46000721a0 msgr2=0x7f460006a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:06.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- 172.21.15.3:0/857604069 shutdown_connections 2024-09-13T16:51:06.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- 172.21.15.3:0/857604069 wait complete. 2024-09-13T16:51:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 Processor -- start 2024-09-13T16:51:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- start start 2024-09-13T16:51:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000b0cc0 0x7f4600147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46000cd390 0x7f46000cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460015c990 0x7f46000cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46000cb3b0 con 0x7f460015c990 2024-09-13T16:51:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46001482d0 con 0x7f46000cd390 2024-09-13T16:51:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.245+0000 7f45ffe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460015de70 con 0x7f46000b0cc0 2024-09-13T16:51:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460015c990 0x7f46000cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460015c990 0x7f46000cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32936/0 (socket says 172.21.15.3:32936) 2024-09-13T16:51:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 -- 172.21.15.3:0/727749083 learned_addr learned my addr 172.21.15.3:0/727749083 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45fea00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000b0cc0 0x7f4600147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45fe000700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46000cd390 0x7f46000cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 -- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000b0cc0 msgr2=0x7f4600147d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000b0cc0 0x7f4600147d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 -- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46000cd390 msgr2=0x7f46000cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46000cd390 0x7f46000cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 -- 172.21.15.3:0/727749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f00094b0 con 0x7f460015c990 2024-09-13T16:51:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45fea00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000b0cc0 0x7f4600147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ff400700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460015c990 0x7f46000cf520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f45f800cc50 tx=0x7f45f800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45f6a00700 1 -- 172.21.15.3:0/727749083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8003c20 con 0x7f460015c990 2024-09-13T16:51:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45f6a00700 1 -- 172.21.15.3:0/727749083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f8003d80 con 0x7f460015c990 2024-09-13T16:51:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460015e150 con 0x7f460015c990 2024-09-13T16:51:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45f6a00700 1 -- 172.21.15.3:0/727749083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800aab0 con 0x7f460015c990 2024-09-13T16:51:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.249+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460015e710 con 0x7f460015c990 2024-09-13T16:51:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.253+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46000cb4f0 con 0x7f460015c990 2024-09-13T16:51:06.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.253+0000 7f45f6a00700 1 -- 172.21.15.3:0/727749083 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f45f800ac10 con 0x7f460015c990 2024-09-13T16:51:06.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.253+0000 7f45f6a00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f45e0066730 0x7f45e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.253+0000 7f45f6a00700 1 -- 172.21.15.3:0/727749083 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f45f8084490 con 0x7f460015c990 2024-09-13T16:51:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.257+0000 7f45fea00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f45e0066730 0x7f45e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.257+0000 7f45fea00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f45e0066730 0x7f45e0068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45f00090b0 tx=0x7f45f001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.261+0000 7f45f6a00700 1 -- 172.21.15.3:0/727749083 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f46000cb4f0 con 0x7f460015c990 2024-09-13T16:51:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.441+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46000cb4f0 con 0x7f460015c990 2024-09-13T16:51:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.441+0000 7f45f6a00700 1 -- 172.21.15.3:0/727749083 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46000cb4f0 con 0x7f460015c990 2024-09-13T16:51:06.447 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:06.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f45e0066730 msgr2=0x7f45e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f45e0066730 0x7f45e0068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45f00090b0 tx=0x7f45f001c040 comp rx=0 tx=0).stop 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460015c990 msgr2=0x7f46000cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460015c990 0x7f46000cf520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f45f800cc50 tx=0x7f45f800cf60 comp rx=0 tx=0).stop 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 shutdown_connections 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f45e0066730 0x7f45e0068bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460015c990 0x7f46000cf520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46000cd390 0x7f46000cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 --2- 172.21.15.3:0/727749083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000b0cc0 0x7f4600147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 >> 172.21.15.3:0/727749083 conn(0x7f46000721a0 msgr2=0x7f460006e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 shutdown_connections 2024-09-13T16:51:06.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.445+0000 7f45ffe00700 1 -- 172.21.15.3:0/727749083 wait complete. 2024-09-13T16:51:06.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:06.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.649+0000 7f132e400700 1 Processor -- start 2024-09-13T16:51:06.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.649+0000 7f132e400700 1 -- start start 2024-09-13T16:51:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132e400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13281444f0 0x7f13280d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132e400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132e400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280b5a30 con 0x7f13280d0680 2024-09-13T16:51:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132e400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13280b5d10 con 0x7f13281444f0 2024-09-13T16:51:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132e400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13280cd5b0 con 0x7f132815ed90 2024-09-13T16:51:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132da00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132da00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:36584/0 (socket says 172.21.15.3:36584) 2024-09-13T16:51:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132da00700 1 -- 172.21.15.3:0/1806475585 learned_addr learned my addr 172.21.15.3:0/1806475585 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132d000700 1 --2- 172.21.15.3:0/1806475585 >> v2:172.21.15.157:3300/0 conn(0x7f13281444f0 0x7f13280d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132da00700 1 -- 172.21.15.3:0/1806475585 >> v2:172.21.15.157:3300/0 conn(0x7f13281444f0 msgr2=0x7f13280d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132da00700 1 --2- 172.21.15.3:0/1806475585 >> v2:172.21.15.157:3300/0 conn(0x7f13281444f0 0x7f13280d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132da00700 1 -- 172.21.15.3:0/1806475585 >> v1:172.21.15.157:6789/0 conn(0x7f13280d0680 legacy=0x7f132815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132da00700 1 -- 172.21.15.3:0/1806475585 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280cd890 con 0x7f132815ed90 2024-09-13T16:51:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f132da00700 1 --2- 172.21.15.3:0/1806475585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280cddb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1320009e50 tx=0x7f132000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac99b07c89bf77b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f1327400700 1 -- 172.21.15.3:0/1806475585 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000e070 con 0x7f132815ed90 2024-09-13T16:51:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f1327400700 1 -- 172.21.15.3:0/1806475585 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132000b910 con 0x7f132815ed90 2024-09-13T16:51:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.653+0000 7f1327400700 1 -- 172.21.15.3:0/1806475585 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000bcc0 con 0x7f132815ed90 2024-09-13T16:51:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 -- 172.21.15.3:0/1806475585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 msgr2=0x7f13280cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 --2- 172.21.15.3:0/1806475585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280cddb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1320009e50 tx=0x7f132000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 -- 172.21.15.3:0/1806475585 shutdown_connections 2024-09-13T16:51:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 --2- 172.21.15.3:0/1806475585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280cddb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 --2- 172.21.15.3:0/1806475585 >> v2:172.21.15.157:3300/0 conn(0x7f13281444f0 0x7f13280d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 -- 172.21.15.3:0/1806475585 >> 172.21.15.3:0/1806475585 conn(0x7f13280721a0 msgr2=0x7f132806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 -- 172.21.15.3:0/1806475585 shutdown_connections 2024-09-13T16:51:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 -- 172.21.15.3:0/1806475585 wait complete. 2024-09-13T16:51:06.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 Processor -- start 2024-09-13T16:51:06.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.657+0000 7f132e400700 1 -- start start 2024-09-13T16:51:06.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132e400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280d0680 0x7f132813d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132e400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13281444f0 0x7f13280b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132e400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132e400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132813ef90 con 0x7f13281444f0 2024-09-13T16:51:06.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132e400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132813ffa0 con 0x7f13280d0680 2024-09-13T16:51:06.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132e400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132813f1e0 con 0x7f132815ed90 2024-09-13T16:51:06.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13281444f0 0x7f13280b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13281444f0 0x7f13280b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32942/0 (socket says 172.21.15.3:32942) 2024-09-13T16:51:06.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 -- 172.21.15.3:0/1949387301 learned_addr learned my addr 172.21.15.3:0/1949387301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:06.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132d000700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280d0680 0x7f132813d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132da00700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 -- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 msgr2=0x7f13280b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 -- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280d0680 msgr2=0x7f132813d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280d0680 0x7f132813d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 -- 172.21.15.3:0/1949387301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320009b30 con 0x7f13281444f0 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132da00700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1327e00700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13281444f0 0x7f13280b26d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f131400c4e0 tx=0x7f131400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1324c00700 1 -- 172.21.15.3:0/1949387301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140052d0 con 0x7f13281444f0 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1324c00700 1 -- 172.21.15.3:0/1949387301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1314005430 con 0x7f13281444f0 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132813f4c0 con 0x7f13281444f0 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f1324c00700 1 -- 172.21.15.3:0/1949387301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140055a0 con 0x7f13281444f0 2024-09-13T16:51:06.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.661+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132813fa80 con 0x7f13281444f0 2024-09-13T16:51:06.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.665+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13280664a0 con 0x7f13281444f0 2024-09-13T16:51:06.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.665+0000 7f1324c00700 1 -- 172.21.15.3:0/1949387301 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1314003900 con 0x7f13281444f0 2024-09-13T16:51:06.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.669+0000 7f1324c00700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:06.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.669+0000 7f1324c00700 1 -- 172.21.15.3:0/1949387301 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1314083890 con 0x7f13281444f0 2024-09-13T16:51:06.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.669+0000 7f132d000700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:06.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.669+0000 7f132d000700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f131c009bf0 tx=0x7f131c009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.673+0000 7f1324c00700 1 -- 172.21.15.3:0/1949387301 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1314050900 con 0x7f13281444f0 2024-09-13T16:51:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.857+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f13280661c0 con 0x7f13281444f0 2024-09-13T16:51:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.857+0000 7f1324c00700 1 -- 172.21.15.3:0/1949387301 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1314053f20 con 0x7f13281444f0 2024-09-13T16:51:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 msgr2=0x7f1308068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f131c009bf0 tx=0x7f131c009240 comp rx=0 tx=0).stop 2024-09-13T16:51:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13281444f0 msgr2=0x7f13280b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13281444f0 0x7f13280b26d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f131400c4e0 tx=0x7f131400c7f0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 shutdown_connections 2024-09-13T16:51:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132815ed90 0x7f13280b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13281444f0 0x7f13280b26d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 --2- 172.21.15.3:0/1949387301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280d0680 0x7f132813d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 >> 172.21.15.3:0/1949387301 conn(0x7f13280721a0 msgr2=0x7f1328142510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 shutdown_connections 2024-09-13T16:51:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:06.861+0000 7f132e400700 1 -- 172.21.15.3:0/1949387301 wait complete. 2024-09-13T16:51:06.887 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf25600700 1 Processor -- start 2024-09-13T16:51:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf25600700 1 -- start start 2024-09-13T16:51:07.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf25600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf2006fff0 0x7faf200b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf25600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faf2015d4d0 0x7faf2015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf25600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf201617e0 con 0x7faf200b3f40 2024-09-13T16:51:07.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf25600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2015e2a0 con 0x7faf2015d4d0 2024-09-13T16:51:07.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf25600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf20160a20 con 0x7faf2006fff0 2024-09-13T16:51:07.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf24c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faf2015d4d0 0x7faf2015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf24c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faf2015d4d0 0x7faf2015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32958/0 (socket says 172.21.15.3:32958) 2024-09-13T16:51:07.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.053+0000 7faf24c00700 1 -- 172.21.15.3:0/1461130561 learned_addr learned my addr 172.21.15.3:0/1461130561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:07.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf24c00700 1 -- 172.21.15.3:0/1461130561 >> v2:172.21.15.157:3300/0 conn(0x7faf2006fff0 msgr2=0x7faf200b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:07.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf24c00700 1 --2- 172.21.15.3:0/1461130561 >> v2:172.21.15.157:3300/0 conn(0x7faf2006fff0 0x7faf200b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf24c00700 1 -- 172.21.15.3:0/1461130561 >> v1:172.21.15.157:6789/0 conn(0x7faf200b3f40 legacy=0x7faf2006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:07.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf24c00700 1 -- 172.21.15.3:0/1461130561 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20160d00 con 0x7faf2015d4d0 2024-09-13T16:51:07.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf24c00700 1 --2- 172.21.15.3:0/1461130561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf2015d4d0 0x7faf2015d8d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf20070b50 tx=0x7faf1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8188cfc173de2133 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf1ea00700 1 -- 172.21.15.3:0/1461130561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800e070 con 0x7faf2015d4d0 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf1ea00700 1 -- 172.21.15.3:0/1461130561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf18003a40 con 0x7faf2015d4d0 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf1ea00700 1 -- 172.21.15.3:0/1461130561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18003df0 con 0x7faf2015d4d0 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf25600700 1 -- 172.21.15.3:0/1461130561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf2015d4d0 msgr2=0x7faf2015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf25600700 1 --2- 172.21.15.3:0/1461130561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf2015d4d0 0x7faf2015d8d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf20070b50 tx=0x7faf1800b330 comp rx=0 tx=0).stop 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf25600700 1 -- 172.21.15.3:0/1461130561 shutdown_connections 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf25600700 1 --2- 172.21.15.3:0/1461130561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf2015d4d0 0x7faf2015d8d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf25600700 1 --2- 172.21.15.3:0/1461130561 >> v2:172.21.15.157:3300/0 conn(0x7faf2006fff0 0x7faf200b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf25600700 1 -- 172.21.15.3:0/1461130561 >> 172.21.15.3:0/1461130561 conn(0x7faf200b2910 msgr2=0x7faf200b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf25600700 1 -- 172.21.15.3:0/1461130561 shutdown_connections 2024-09-13T16:51:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.057+0000 7faf25600700 1 -- 172.21.15.3:0/1461130561 wait complete. 2024-09-13T16:51:07.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 Processor -- start 2024-09-13T16:51:07.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 -- start start 2024-09-13T16:51:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2006fff0 0x7faf201f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf200b3f40 0x7faf20156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf2015d4d0 0x7faf20143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf201420b0 con 0x7faf200b3f40 2024-09-13T16:51:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20141710 con 0x7faf2006fff0 2024-09-13T16:51:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf201419f0 con 0x7faf2015d4d0 2024-09-13T16:51:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf200b3f40 0x7faf20156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf200b3f40 0x7faf20156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32972/0 (socket says 172.21.15.3:32972) 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 -- 172.21.15.3:0/4250589157 learned_addr learned my addr 172.21.15.3:0/4250589157 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1fe00700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2006fff0 0x7faf201f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf24c00700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf2015d4d0 0x7faf20143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 -- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf2015d4d0 msgr2=0x7faf20143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf2015d4d0 0x7faf20143970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 -- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2006fff0 msgr2=0x7faf201f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2006fff0 0x7faf201f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 -- 172.21.15.3:0/4250589157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf18009b30 con 0x7faf200b3f40 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf1f400700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf200b3f40 0x7faf20156570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b550 tx=0x7faf0c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf17e00700 1 -- 172.21.15.3:0/4250589157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c011830 con 0x7faf200b3f40 2024-09-13T16:51:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf17e00700 1 -- 172.21.15.3:0/4250589157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c011990 con 0x7faf200b3f40 2024-09-13T16:51:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf17e00700 1 -- 172.21.15.3:0/4250589157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c011d40 con 0x7faf200b3f40 2024-09-13T16:51:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.061+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf20141cd0 con 0x7faf200b3f40 2024-09-13T16:51:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.065+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf20156ab0 con 0x7faf200b3f40 2024-09-13T16:51:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.065+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf201468d0 con 0x7faf200b3f40 2024-09-13T16:51:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.065+0000 7faf17e00700 1 -- 172.21.15.3:0/4250589157 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faf0c01a400 con 0x7faf200b3f40 2024-09-13T16:51:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.069+0000 7faf17e00700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7faf00066730 0x7faf00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.069+0000 7faf17e00700 1 -- 172.21.15.3:0/4250589157 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faf0c084340 con 0x7faf200b3f40 2024-09-13T16:51:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.069+0000 7faf1fe00700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7faf00066730 0x7faf00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.069+0000 7faf1fe00700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7faf00066730 0x7faf00068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf10005150 tx=0x7faf10007ee0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.077+0000 7faf17e00700 1 -- 172.21.15.3:0/4250589157 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faf0c051450 con 0x7faf200b3f40 2024-09-13T16:51:07.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.305+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf200b4320 con 0x7faf200b3f40 2024-09-13T16:51:07.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.305+0000 7faf17e00700 1 -- 172.21.15.3:0/4250589157 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4727 (secure 0 0 0) 0x7faf0c059e60 con 0x7faf200b3f40 2024-09-13T16:51:07.312 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:07.312 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:55.700655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:51:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7faf00066730 msgr2=0x7faf00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7faf00066730 0x7faf00068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf10005150 tx=0x7faf10007ee0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf200b3f40 msgr2=0x7faf20156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf200b3f40 0x7faf20156570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b550 tx=0x7faf0c00b860 comp rx=0 tx=0).stop 2024-09-13T16:51:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 shutdown_connections 2024-09-13T16:51:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7faf00066730 0x7faf00068bf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf2015d4d0 0x7faf20143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf200b3f40 0x7faf20156570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 --2- 172.21.15.3:0/4250589157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2006fff0 0x7faf201f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 >> 172.21.15.3:0/4250589157 conn(0x7faf200b2910 msgr2=0x7faf200b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:07.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 shutdown_connections 2024-09-13T16:51:07.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.309+0000 7faf25600700 1 -- 172.21.15.3:0/4250589157 wait complete. 2024-09-13T16:51:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-13T16:51:07.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:07.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcd0a00700 1 Processor -- start 2024-09-13T16:51:07.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcd0a00700 1 -- start start 2024-09-13T16:51:07.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcd0a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc144790 0x7fdccc06c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcd0a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdccc06cb20 0x7fdccc15b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcd0a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0b4100 con 0x7fdccc1443b0 2024-09-13T16:51:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcd0a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdccc06cf90 con 0x7fdccc06cb20 2024-09-13T16:51:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcd0a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0ce8e0 con 0x7fdccc144790 2024-09-13T16:51:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdccaa00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc144790 0x7fdccc06c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdccb400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdccc1443b0 0x7fdccc0b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:49004/0 (socket says 172.21.15.3:49004) 2024-09-13T16:51:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdccbe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdccc06cb20 0x7fdccc15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdccb400700 1 -- 172.21.15.3:0/1990397876 learned_addr learned my addr 172.21.15.3:0/1990397876 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcca000700 1 -- 172.21.15.3:0/1990397876 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388518440 0 0) 0x7fdccc0b4100 con 0x7fdccc1443b0 2024-09-13T16:51:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.493+0000 7fdcca000700 1 -- 172.21.15.3:0/1990397876 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb00035d0 con 0x7fdccc1443b0 2024-09-13T16:51:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdccbe00700 1 -- 172.21.15.3:0/1990397876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc144790 msgr2=0x7fdccc06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdccbe00700 1 --2- 172.21.15.3:0/1990397876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc144790 0x7fdccc06c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdccbe00700 1 -- 172.21.15.3:0/1990397876 >> v1:172.21.15.157:6789/0 conn(0x7fdccc1443b0 legacy=0x7fdccc0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdccbe00700 1 -- 172.21.15.3:0/1990397876 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc0ccd30 con 0x7fdccc06cb20 2024-09-13T16:51:07.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdccbe00700 1 --2- 172.21.15.3:0/1990397876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc06cb20 0x7fdccc15b220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009d90 tx=0x7fdcc400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91908bd70b8de0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcca000700 1 -- 172.21.15.3:0/1990397876 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400e070 con 0x7fdccc06cb20 2024-09-13T16:51:07.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcca000700 1 -- 172.21.15.3:0/1990397876 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc400bae0 con 0x7fdccc06cb20 2024-09-13T16:51:07.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcca000700 1 -- 172.21.15.3:0/1990397876 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400be90 con 0x7fdccc06cb20 2024-09-13T16:51:07.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcd0a00700 1 -- 172.21.15.3:0/1990397876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc06cb20 msgr2=0x7fdccc15b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/1990397876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc06cb20 0x7fdccc15b220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009d90 tx=0x7fdcc400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcd0a00700 1 -- 172.21.15.3:0/1990397876 shutdown_connections 2024-09-13T16:51:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/1990397876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc06cb20 0x7fdccc15b220 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/1990397876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc144790 0x7fdccc06c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcd0a00700 1 -- 172.21.15.3:0/1990397876 >> 172.21.15.3:0/1990397876 conn(0x7fdccc0b0cf0 msgr2=0x7fdccc0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcd0a00700 1 -- 172.21.15.3:0/1990397876 shutdown_connections 2024-09-13T16:51:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.497+0000 7fdcd0a00700 1 -- 172.21.15.3:0/1990397876 wait complete. 2024-09-13T16:51:07.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.501+0000 7fdcd0a00700 1 Processor -- start 2024-09-13T16:51:07.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.501+0000 7fdcd0a00700 1 -- start start 2024-09-13T16:51:07.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcd0a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc06cb20 0x7fdccc137440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcd0a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc1443b0 0x7fdccc13beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcd0a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc13c3f0 0x7fdccc13df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcd0a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc13c8a0 con 0x7fdccc13c3f0 2024-09-13T16:51:07.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcd0a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc137980 con 0x7fdccc1443b0 2024-09-13T16:51:07.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcd0a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc134b80 con 0x7fdccc06cb20 2024-09-13T16:51:07.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccb400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc06cb20 0x7fdccc137440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc13c3f0 0x7fdccc13df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc13c3f0 0x7fdccc13df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32980/0 (socket says 172.21.15.3:32980) 2024-09-13T16:51:07.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccbe00700 1 -- 172.21.15.3:0/901721902 learned_addr learned my addr 172.21.15.3:0/901721902 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:07.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccaa00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc1443b0 0x7fdccc13beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccb400700 1 -- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc1443b0 msgr2=0x7fdccc13beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccb400700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc1443b0 0x7fdccc13beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccb400700 1 -- 172.21.15.3:0/901721902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc13c3f0 msgr2=0x7fdccc13df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccb400700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc13c3f0 0x7fdccc13df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccb400700 1 -- 172.21.15.3:0/901721902 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4009a70 con 0x7fdccc06cb20 2024-09-13T16:51:07.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccaa00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc1443b0 0x7fdccc13beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:07.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdccb400700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc06cb20 0x7fdccc137440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0011e0 tx=0x7fdcbc00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcc3400700 1 -- 172.21.15.3:0/901721902 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc004ca0 con 0x7fdccc06cb20 2024-09-13T16:51:07.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcc3400700 1 -- 172.21.15.3:0/901721902 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc004e00 con 0x7fdccc06cb20 2024-09-13T16:51:07.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcc3400700 1 -- 172.21.15.3:0/901721902 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc007310 con 0x7fdccc06cb20 2024-09-13T16:51:07.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc0703b0 con 0x7fdccc06cb20 2024-09-13T16:51:07.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.505+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc13efb0 con 0x7fdccc06cb20 2024-09-13T16:51:07.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.509+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc13f1d0 con 0x7fdccc06cb20 2024-09-13T16:51:07.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.509+0000 7fdcc3400700 1 -- 172.21.15.3:0/901721902 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdcbc007470 con 0x7fdccc06cb20 2024-09-13T16:51:07.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.509+0000 7fdcc3400700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac066730 0x7fdcac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.509+0000 7fdcc3400700 1 -- 172.21.15.3:0/901721902 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdcbc084180 con 0x7fdccc06cb20 2024-09-13T16:51:07.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.509+0000 7fdccaa00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac066730 0x7fdcac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.509+0000 7fdccaa00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac066730 0x7fdcac068bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdccc141220 tx=0x7fdcb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.517+0000 7fdcc3400700 1 -- 172.21.15.3:0/901721902 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdcbc0510d0 con 0x7fdccc06cb20 2024-09-13T16:51:07.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.705+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdccc0cca20 con 0x7fdccc06cb20 2024-09-13T16:51:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.709+0000 7fdcc3400700 1 -- 172.21.15.3:0/901721902 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7fdcbc0546f0 con 0x7fdccc06cb20 2024-09-13T16:51:07.714 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:07.714 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac066730 msgr2=0x7fdcac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac066730 0x7fdcac068bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdccc141220 tx=0x7fdcb8008040 comp rx=0 tx=0).stop 2024-09-13T16:51:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc06cb20 msgr2=0x7fdccc137440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc06cb20 0x7fdccc137440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0011e0 tx=0x7fdcbc00c800 comp rx=0 tx=0).stop 2024-09-13T16:51:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 shutdown_connections 2024-09-13T16:51:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac066730 0x7fdcac068bf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc13c3f0 0x7fdccc13df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc1443b0 0x7fdccc13beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/901721902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc06cb20 0x7fdccc137440 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 >> 172.21.15.3:0/901721902 conn(0x7fdccc0b0cf0 msgr2=0x7fdccc0afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 shutdown_connections 2024-09-13T16:51:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.713+0000 7fdcd0a00700 1 -- 172.21.15.3:0/901721902 wait complete. 2024-09-13T16:51:07.741 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:07.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:07.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8093e00700 1 Processor -- start 2024-09-13T16:51:07.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8093e00700 1 -- start start 2024-09-13T16:51:07.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8093e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8093e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f809415cfe0 0x7f80940b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8093e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809406a5a0 con 0x7f8094160970 2024-09-13T16:51:07.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8093e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8094160e70 con 0x7f809415cfe0 2024-09-13T16:51:07.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8093e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8094161150 con 0x7f809415cc00 2024-09-13T16:51:07.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8092000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f809415cfe0 0x7f80940b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8092000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f809415cfe0 0x7f80940b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32992/0 (socket says 172.21.15.3:32992) 2024-09-13T16:51:07.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.921+0000 7f8092000700 1 -- 172.21.15.3:0/3699876049 learned_addr learned my addr 172.21.15.3:0/3699876049 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:07.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8092a00700 1 --2- 172.21.15.3:0/3699876049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8092000700 1 -- 172.21.15.3:0/3699876049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 msgr2=0x7f809415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:07.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8092000700 1 --2- 172.21.15.3:0/3699876049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8092000700 1 -- 172.21.15.3:0/3699876049 >> v1:172.21.15.157:6789/0 conn(0x7f8094160970 legacy=0x7f80940719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8092000700 1 -- 172.21.15.3:0/3699876049 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094161430 con 0x7f809415cfe0 2024-09-13T16:51:07.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8092000700 1 --2- 172.21.15.3:0/3699876049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f80940b0260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8080009910 tx=0x7f8080009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735c52ee9721c304 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8091600700 1 -- 172.21.15.3:0/3699876049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000ba60 con 0x7f809415cfe0 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8091600700 1 -- 172.21.15.3:0/3699876049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808000bbc0 con 0x7f809415cfe0 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8091600700 1 -- 172.21.15.3:0/3699876049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080009e60 con 0x7f809415cfe0 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8093e00700 1 -- 172.21.15.3:0/3699876049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 msgr2=0x7f80940b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8093e00700 1 --2- 172.21.15.3:0/3699876049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f80940b0260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8080009910 tx=0x7f8080009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8093e00700 1 -- 172.21.15.3:0/3699876049 shutdown_connections 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8093e00700 1 --2- 172.21.15.3:0/3699876049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f80940b0260 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8093e00700 1 --2- 172.21.15.3:0/3699876049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8093e00700 1 -- 172.21.15.3:0/3699876049 >> 172.21.15.3:0/3699876049 conn(0x7f80940b2d50 msgr2=0x7f80940b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8093e00700 1 -- 172.21.15.3:0/3699876049 shutdown_connections 2024-09-13T16:51:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.925+0000 7f8093e00700 1 -- 172.21.15.3:0/3699876049 wait complete. 2024-09-13T16:51:07.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 Processor -- start 2024-09-13T16:51:07.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 -- start start 2024-09-13T16:51:07.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f809415eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8094160970 0x7f809414df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809415f410 con 0x7f809415cfe0 2024-09-13T16:51:07.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8094150a70 con 0x7f8094160970 2024-09-13T16:51:07.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809414ed70 con 0x7f809415cc00 2024-09-13T16:51:07.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f809415eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f809415eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32998/0 (socket says 172.21.15.3:32998) 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 -- 172.21.15.3:0/3811081257 learned_addr learned my addr 172.21.15.3:0/3811081257 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 -- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 msgr2=0x7f809415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092a00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093400700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8094160970 0x7f809414df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 -- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8094160970 msgr2=0x7f809414df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8094160970 0x7f809414df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 -- 172.21.15.3:0/3811081257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080009570 con 0x7f809415cfe0 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093400700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8094160970 0x7f809414df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092a00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8092000700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f809415eed0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8080013000 tx=0x7f8080004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f808aa00700 1 -- 172.21.15.3:0/3811081257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080004610 con 0x7f809415cfe0 2024-09-13T16:51:07.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f808aa00700 1 -- 172.21.15.3:0/3811081257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808001f070 con 0x7f809415cfe0 2024-09-13T16:51:07.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809414f050 con 0x7f809415cfe0 2024-09-13T16:51:07.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f808aa00700 1 -- 172.21.15.3:0/3811081257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080003900 con 0x7f809415cfe0 2024-09-13T16:51:07.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.929+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809414f330 con 0x7f809415cfe0 2024-09-13T16:51:07.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.933+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809414f4c0 con 0x7f809415cfe0 2024-09-13T16:51:07.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.933+0000 7f808aa00700 1 -- 172.21.15.3:0/3811081257 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8080003a60 con 0x7f809415cfe0 2024-09-13T16:51:07.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.933+0000 7f808aa00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8074066730 0x7f8074068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:07.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.933+0000 7f808aa00700 1 -- 172.21.15.3:0/3811081257 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8080021070 con 0x7f809415cfe0 2024-09-13T16:51:07.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.933+0000 7f8092a00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8074066730 0x7f8074068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.937+0000 7f8092a00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8074066730 0x7f8074068bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80940adef0 tx=0x7f8084008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:07.937+0000 7f808aa00700 1 -- 172.21.15.3:0/3811081257 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8080052d90 con 0x7f809415cfe0 2024-09-13T16:51:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80941508d0 con 0x7f809415cfe0 2024-09-13T16:51:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f808aa00700 1 -- 172.21.15.3:0/3811081257 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80941508d0 con 0x7f809415cfe0 2024-09-13T16:51:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8074066730 msgr2=0x7f8074068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8074066730 0x7f8074068bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80940adef0 tx=0x7f8084008040 comp rx=0 tx=0).stop 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 msgr2=0x7f809415eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f809415eed0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8080013000 tx=0x7f8080004770 comp rx=0 tx=0).stop 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 shutdown_connections 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8074066730 0x7f8074068bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8094160970 0x7f809414df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f809415cfe0 0x7f809415eed0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 --2- 172.21.15.3:0/3811081257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f809415cc00 0x7f809415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.117+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 >> 172.21.15.3:0/3811081257 conn(0x7f80940b2d50 msgr2=0x7f809406c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.121+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 shutdown_connections 2024-09-13T16:51:08.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.121+0000 7f8093e00700 1 -- 172.21.15.3:0/3811081257 wait complete. 2024-09-13T16:51:08.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:08.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.301+0000 7f95dd400700 1 Processor -- start 2024-09-13T16:51:08.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95dd400700 1 -- start start 2024-09-13T16:51:08.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95dd400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95d81519f0 0x7f95d806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95dd400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d8151dd0 0x7f95d80b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95dd400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d80cf340 con 0x7f95d806f2d0 2024-09-13T16:51:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95dd400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d815e7d0 con 0x7f95d81519f0 2024-09-13T16:51:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95dd400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d815eab0 con 0x7f95d8151dd0 2024-09-13T16:51:08.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95d5e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f95d806f2d0 0x7f95d806f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34648/0 (socket says 172.21.15.3:34648) 2024-09-13T16:51:08.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95d5e00700 1 -- 172.21.15.3:0/3858971895 learned_addr learned my addr 172.21.15.3:0/3858971895 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:08.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95d6800700 1 --2- 172.21.15.3:0/3858971895 >> v2:172.21.15.157:3300/0 conn(0x7f95d81519f0 0x7f95d806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.305+0000 7f95d7200700 1 --2- 172.21.15.3:0/3858971895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d8151dd0 0x7f95d80b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d5400700 1 -- 172.21.15.3:0/3858971895 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083491811 0 0) 0x7f95d80cf340 con 0x7f95d806f2d0 2024-09-13T16:51:08.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d5400700 1 -- 172.21.15.3:0/3858971895 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95bc0035d0 con 0x7f95d806f2d0 2024-09-13T16:51:08.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d7200700 1 -- 172.21.15.3:0/3858971895 >> v2:172.21.15.157:3300/0 conn(0x7f95d81519f0 msgr2=0x7f95d806ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:08.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d7200700 1 --2- 172.21.15.3:0/3858971895 >> v2:172.21.15.157:3300/0 conn(0x7f95d81519f0 0x7f95d806ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d7200700 1 -- 172.21.15.3:0/3858971895 >> v1:172.21.15.3:6789/0 conn(0x7f95d806f2d0 legacy=0x7f95d806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d5400700 1 -- 172.21.15.3:0/3858971895 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944307453 0 0) 0x7f95bc0035d0 con 0x7f95d806f2d0 2024-09-13T16:51:08.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d7200700 1 -- 172.21.15.3:0/3858971895 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d815ed90 con 0x7f95d8151dd0 2024-09-13T16:51:08.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d6800700 1 --2- 172.21.15.3:0/3858971895 >> v2:172.21.15.157:3300/0 conn(0x7f95d81519f0 0x7f95d806ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:08.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d7200700 1 --2- 172.21.15.3:0/3858971895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d8151dd0 0x7f95d80b3e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009d90 tx=0x7f95d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a30c141a5064b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d5400700 1 -- 172.21.15.3:0/3858971895 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000e070 con 0x7f95d8151dd0 2024-09-13T16:51:08.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d5400700 1 -- 172.21.15.3:0/3858971895 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d0003a40 con 0x7f95d8151dd0 2024-09-13T16:51:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95d5400700 1 -- 172.21.15.3:0/3858971895 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4001690 con 0x7f95d8151dd0 2024-09-13T16:51:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95dd400700 1 -- 172.21.15.3:0/3858971895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d8151dd0 msgr2=0x7f95d80b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95dd400700 1 --2- 172.21.15.3:0/3858971895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d8151dd0 0x7f95d80b3e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009d90 tx=0x7f95d000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95dd400700 1 -- 172.21.15.3:0/3858971895 shutdown_connections 2024-09-13T16:51:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95dd400700 1 --2- 172.21.15.3:0/3858971895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d8151dd0 0x7f95d80b3e00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95dd400700 1 --2- 172.21.15.3:0/3858971895 >> v2:172.21.15.157:3300/0 conn(0x7f95d81519f0 0x7f95d806ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95dd400700 1 -- 172.21.15.3:0/3858971895 >> 172.21.15.3:0/3858971895 conn(0x7f95d80b2b00 msgr2=0x7f95d80b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95dd400700 1 -- 172.21.15.3:0/3858971895 shutdown_connections 2024-09-13T16:51:08.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.309+0000 7f95dd400700 1 -- 172.21.15.3:0/3858971895 wait complete. 2024-09-13T16:51:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95dd400700 1 Processor -- start 2024-09-13T16:51:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95dd400700 1 -- start start 2024-09-13T16:51:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95dd400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d806f2d0 0x7f95d814bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95dd400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d81519f0 0x7f95d806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95dd400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d8151dd0 0x7f95d80b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95dd400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d814efe0 con 0x7f95d81519f0 2024-09-13T16:51:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95dd400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d814f2c0 con 0x7f95d8151dd0 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95dd400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d814f5a0 con 0x7f95d806f2d0 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d81519f0 0x7f95d806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d81519f0 0x7f95d806cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33012/0 (socket says 172.21.15.3:33012) 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d5e00700 1 -- 172.21.15.3:0/3809240173 learned_addr learned my addr 172.21.15.3:0/3809240173 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d6800700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d806f2d0 0x7f95d814bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d7200700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d8151dd0 0x7f95d80b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d5e00700 1 -- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d806f2d0 msgr2=0x7f95d814bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d5e00700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d806f2d0 0x7f95d814bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d5e00700 1 -- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d8151dd0 msgr2=0x7f95d80b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d5e00700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d8151dd0 0x7f95d80b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d5e00700 1 -- 172.21.15.3:0/3809240173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80094b0 con 0x7f95d81519f0 2024-09-13T16:51:08.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.313+0000 7f95d6800700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d806f2d0 0x7f95d814bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:08.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.317+0000 7f95d5e00700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d81519f0 0x7f95d806cec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c780 tx=0x7f95c400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.317+0000 7f95cea00700 1 -- 172.21.15.3:0/3809240173 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4004ca0 con 0x7f95d81519f0 2024-09-13T16:51:08.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.317+0000 7f95cea00700 1 -- 172.21.15.3:0/3809240173 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4004e00 con 0x7f95d81519f0 2024-09-13T16:51:08.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.317+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d0009a70 con 0x7f95d81519f0 2024-09-13T16:51:08.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.317+0000 7f95cea00700 1 -- 172.21.15.3:0/3809240173 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4007310 con 0x7f95d81519f0 2024-09-13T16:51:08.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.317+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d814fb60 con 0x7f95d81519f0 2024-09-13T16:51:08.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.317+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d806ed90 con 0x7f95d81519f0 2024-09-13T16:51:08.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.317+0000 7f95cea00700 1 -- 172.21.15.3:0/3809240173 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f95c4007470 con 0x7f95d81519f0 2024-09-13T16:51:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.321+0000 7f95cea00700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95b8066730 0x7f95b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.321+0000 7f95cea00700 1 -- 172.21.15.3:0/3809240173 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f95c4084390 con 0x7f95d81519f0 2024-09-13T16:51:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.321+0000 7f95d6800700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95b8066730 0x7f95b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.321+0000 7f95d6800700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95b8066730 0x7f95b8068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009dd0 tx=0x7f95c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.321+0000 7f95cea00700 1 -- 172.21.15.3:0/3809240173 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f95c4051370 con 0x7f95d81519f0 2024-09-13T16:51:08.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.509+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95d8139fc0 con 0x7f95d81519f0 2024-09-13T16:51:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.509+0000 7f95cea00700 1 -- 172.21.15.3:0/3809240173 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95c4007720 con 0x7f95d81519f0 2024-09-13T16:51:08.515 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95b8066730 msgr2=0x7f95b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95b8066730 0x7f95b8068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009dd0 tx=0x7f95c8008040 comp rx=0 tx=0).stop 2024-09-13T16:51:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d81519f0 msgr2=0x7f95d806cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d81519f0 0x7f95d806cec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c780 tx=0x7f95c400cb40 comp rx=0 tx=0).stop 2024-09-13T16:51:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 shutdown_connections 2024-09-13T16:51:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95b8066730 0x7f95b8068bf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d8151dd0 0x7f95d80b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d81519f0 0x7f95d806cec0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 --2- 172.21.15.3:0/3809240173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d806f2d0 0x7f95d814bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 >> 172.21.15.3:0/3809240173 conn(0x7f95d80b2b00 msgr2=0x7f95d813d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 shutdown_connections 2024-09-13T16:51:08.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.513+0000 7f95dd400700 1 -- 172.21.15.3:0/3809240173 wait complete. 2024-09-13T16:51:08.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:08.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.717+0000 7f6243200700 1 Processor -- start 2024-09-13T16:51:08.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.717+0000 7f6243200700 1 -- start start 2024-09-13T16:51:08.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6243200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62440b5080 0x7f6244167730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6243200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440b2230 con 0x7f624415c990 2024-09-13T16:51:08.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6243200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440b1190 con 0x7f62440cd390 2024-09-13T16:51:08.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6243200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62440b1470 con 0x7f62440b5080 2024-09-13T16:51:08.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6241e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f624415c990 0x7f62440b0780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34650/0 (socket says 172.21.15.3:34650) 2024-09-13T16:51:08.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6241e00700 1 -- 172.21.15.3:0/635130751 learned_addr learned my addr 172.21.15.3:0/635130751 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:08.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6242800700 1 --2- 172.21.15.3:0/635130751 >> v2:172.21.15.157:3300/0 conn(0x7f62440b5080 0x7f6244167730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6240a00700 1 -- 172.21.15.3:0/635130751 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017168429 0 0) 0x7f62440b2230 con 0x7f624415c990 2024-09-13T16:51:08.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6240a00700 1 -- 172.21.15.3:0/635130751 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62280035d0 con 0x7f624415c990 2024-09-13T16:51:08.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6240a00700 1 -- 172.21.15.3:0/635130751 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262826134 0 0) 0x7f62440b1190 con 0x7f62440cd390 2024-09-13T16:51:08.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6240a00700 1 -- 172.21.15.3:0/635130751 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62440b2230 con 0x7f62440cd390 2024-09-13T16:51:08.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6242800700 1 -- 172.21.15.3:0/635130751 >> v1:172.21.15.157:6789/0 conn(0x7f62440cd390 legacy=0x7f62440b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6242800700 1 -- 172.21.15.3:0/635130751 >> v1:172.21.15.3:6789/0 conn(0x7f624415c990 legacy=0x7f62440b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6242800700 1 -- 172.21.15.3:0/635130751 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440b1750 con 0x7f62440b5080 2024-09-13T16:51:08.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.721+0000 7f6242800700 1 --2- 172.21.15.3:0/635130751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440b5080 0x7f6244167730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f623c009d90 tx=0x7f623c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da515f933f718b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6240a00700 1 -- 172.21.15.3:0/635130751 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00e070 con 0x7f62440b5080 2024-09-13T16:51:08.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6240a00700 1 -- 172.21.15.3:0/635130751 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c00bb20 con 0x7f62440b5080 2024-09-13T16:51:08.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6240a00700 1 -- 172.21.15.3:0/635130751 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c012430 con 0x7f62440b5080 2024-09-13T16:51:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 -- 172.21.15.3:0/635130751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440b5080 msgr2=0x7f6244167730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 --2- 172.21.15.3:0/635130751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440b5080 0x7f6244167730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f623c009d90 tx=0x7f623c00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 -- 172.21.15.3:0/635130751 shutdown_connections 2024-09-13T16:51:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 --2- 172.21.15.3:0/635130751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440b5080 0x7f6244167730 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 -- 172.21.15.3:0/635130751 >> 172.21.15.3:0/635130751 conn(0x7f62440721a0 msgr2=0x7f624406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 -- 172.21.15.3:0/635130751 shutdown_connections 2024-09-13T16:51:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 -- 172.21.15.3:0/635130751 wait complete. 2024-09-13T16:51:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 Processor -- start 2024-09-13T16:51:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.725+0000 7f6243200700 1 -- start start 2024-09-13T16:51:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6243200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62440b5080 0x7f624414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6243200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440cd390 0x7f624414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6243200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f624415c990 0x7f6244155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6243200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624414f470 con 0x7f624415c990 2024-09-13T16:51:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6243200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244154480 con 0x7f62440cd390 2024-09-13T16:51:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6243200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624414f660 con 0x7f62440b5080 2024-09-13T16:51:08.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6242800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f624415c990 0x7f6244155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6242800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f624415c990 0x7f6244155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33026/0 (socket says 172.21.15.3:33026) 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6242800700 1 -- 172.21.15.3:0/2602098403 learned_addr learned my addr 172.21.15.3:0/2602098403 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6241e00700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62440b5080 0x7f624414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6241400700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440cd390 0x7f624414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6241e00700 1 -- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440cd390 msgr2=0x7f624414ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6241e00700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440cd390 0x7f624414ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6241e00700 1 -- 172.21.15.3:0/2602098403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f624415c990 msgr2=0x7f6244155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6241e00700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f624415c990 0x7f6244155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6241e00700 1 -- 172.21.15.3:0/2602098403 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009a70 con 0x7f62440b5080 2024-09-13T16:51:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6241e00700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62440b5080 0x7f624414df60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62340011e0 tx=0x7f623400c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f623a000700 1 -- 172.21.15.3:0/2602098403 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234005b90 con 0x7f62440b5080 2024-09-13T16:51:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f623a000700 1 -- 172.21.15.3:0/2602098403 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6234005cf0 con 0x7f62440b5080 2024-09-13T16:51:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f623a000700 1 -- 172.21.15.3:0/2602098403 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234007310 con 0x7f62440b5080 2024-09-13T16:51:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6244154690 con 0x7f62440b5080 2024-09-13T16:51:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.729+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6244154c50 con 0x7f62440b5080 2024-09-13T16:51:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.733+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624415cd70 con 0x7f62440b5080 2024-09-13T16:51:08.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.733+0000 7f623a000700 1 -- 172.21.15.3:0/2602098403 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6234007470 con 0x7f62440b5080 2024-09-13T16:51:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.737+0000 7f623a000700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6224066780 0x7f6224068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.737+0000 7f623a000700 1 -- 172.21.15.3:0/2602098403 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6234083c40 con 0x7f62440b5080 2024-09-13T16:51:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.737+0000 7f6241400700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6224066780 0x7f6224068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.737+0000 7f6241400700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6224066780 0x7f6224068c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6244160370 tx=0x7f6230009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.745+0000 7f623a000700 1 -- 172.21.15.3:0/2602098403 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6234050d30 con 0x7f62440b5080 2024-09-13T16:51:08.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.925+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62441565b0 con 0x7f62440b5080 2024-09-13T16:51:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.925+0000 7f623a000700 1 -- 172.21.15.3:0/2602098403 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6234005e60 con 0x7f62440b5080 2024-09-13T16:51:08.930 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6224066780 msgr2=0x7f6224068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6224066780 0x7f6224068c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6244160370 tx=0x7f6230009040 comp rx=0 tx=0).stop 2024-09-13T16:51:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62440b5080 msgr2=0x7f624414df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62440b5080 0x7f624414df60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62340011e0 tx=0x7f623400c700 comp rx=0 tx=0).stop 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 shutdown_connections 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6224066780 0x7f6224068c40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f624415c990 0x7f6244155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62440cd390 0x7f624414ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 --2- 172.21.15.3:0/2602098403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62440b5080 0x7f624414df60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 >> 172.21.15.3:0/2602098403 conn(0x7f62440721a0 msgr2=0x7f624406d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 shutdown_connections 2024-09-13T16:51:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:08.929+0000 7f6243200700 1 -- 172.21.15.3:0/2602098403 wait complete. 2024-09-13T16:51:08.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:09.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.149+0000 7fe893200700 1 Processor -- start 2024-09-13T16:51:09.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.149+0000 7fe893200700 1 -- start start 2024-09-13T16:51:09.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe893200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8940b2ce0 0x7fe894159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe893200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89406cba0 con 0x7fe8941507b0 2024-09-13T16:51:09.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe893200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89406ce80 con 0x7fe89406aaa0 2024-09-13T16:51:09.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe893200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89406d160 con 0x7fe8940b2ce0 2024-09-13T16:51:09.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe891e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe8941507b0 0x7fe89406a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34664/0 (socket says 172.21.15.3:34664) 2024-09-13T16:51:09.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe891e00700 1 -- 172.21.15.3:0/1178348293 learned_addr learned my addr 172.21.15.3:0/1178348293 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:09.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe892800700 1 --2- 172.21.15.3:0/1178348293 >> v2:172.21.15.3:3300/0 conn(0x7fe8940b2ce0 0x7fe894159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe890a00700 1 -- 172.21.15.3:0/1178348293 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670159510 0 0) 0x7fe89406cba0 con 0x7fe8941507b0 2024-09-13T16:51:09.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe890a00700 1 -- 172.21.15.3:0/1178348293 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8780035d0 con 0x7fe8941507b0 2024-09-13T16:51:09.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe890a00700 1 -- 172.21.15.3:0/1178348293 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771120650 0 0) 0x7fe89406ce80 con 0x7fe89406aaa0 2024-09-13T16:51:09.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe892800700 1 -- 172.21.15.3:0/1178348293 >> v1:172.21.15.157:6789/0 conn(0x7fe89406aaa0 legacy=0x7fe89415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe892800700 1 -- 172.21.15.3:0/1178348293 >> v1:172.21.15.3:6789/0 conn(0x7fe8941507b0 legacy=0x7fe89406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe892800700 1 -- 172.21.15.3:0/1178348293 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89406d440 con 0x7fe8940b2ce0 2024-09-13T16:51:09.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe892800700 1 --2- 172.21.15.3:0/1178348293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8940b2ce0 0x7fe894159e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009d90 tx=0x7fe88c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a6afd65c2c59e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:09.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe890a00700 1 -- 172.21.15.3:0/1178348293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00e070 con 0x7fe8940b2ce0 2024-09-13T16:51:09.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe890a00700 1 -- 172.21.15.3:0/1178348293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c00bb20 con 0x7fe8940b2ce0 2024-09-13T16:51:09.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.153+0000 7fe890a00700 1 -- 172.21.15.3:0/1178348293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00be90 con 0x7fe8940b2ce0 2024-09-13T16:51:09.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.157+0000 7fe893200700 1 -- 172.21.15.3:0/1178348293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8940b2ce0 msgr2=0x7fe894159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.157+0000 7fe893200700 1 --2- 172.21.15.3:0/1178348293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8940b2ce0 0x7fe894159e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009d90 tx=0x7fe88c00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:09.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.157+0000 7fe893200700 1 -- 172.21.15.3:0/1178348293 shutdown_connections 2024-09-13T16:51:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.157+0000 7fe893200700 1 --2- 172.21.15.3:0/1178348293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8940b2ce0 0x7fe894159e20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.157+0000 7fe893200700 1 -- 172.21.15.3:0/1178348293 >> 172.21.15.3:0/1178348293 conn(0x7fe8940af620 msgr2=0x7fe8940afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.157+0000 7fe893200700 1 -- 172.21.15.3:0/1178348293 shutdown_connections 2024-09-13T16:51:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.157+0000 7fe893200700 1 -- 172.21.15.3:0/1178348293 wait complete. 2024-09-13T16:51:09.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe893200700 1 Processor -- start 2024-09-13T16:51:09.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe893200700 1 -- start start 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe893200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe89406aaa0 0x7fe894150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe893200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940b2ce0 0x7fe89415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe893200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941507b0 0x7fe8940705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe893200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8941535c0 con 0x7fe89406aaa0 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe893200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894152980 con 0x7fe8941507b0 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe893200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe894151500 con 0x7fe8940b2ce0 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe891e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe89406aaa0 0x7fe894150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe891e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe89406aaa0 0x7fe894150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33046/0 (socket says 172.21.15.3:33046) 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe891e00700 1 -- 172.21.15.3:0/2172733571 learned_addr learned my addr 172.21.15.3:0/2172733571 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe891400700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940b2ce0 0x7fe89415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.161+0000 7fe892800700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941507b0 0x7fe8940705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe891e00700 1 -- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940b2ce0 msgr2=0x7fe89415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe891e00700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940b2ce0 0x7fe89415cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe891e00700 1 -- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941507b0 msgr2=0x7fe8940705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe891e00700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941507b0 0x7fe8940705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe891e00700 1 -- 172.21.15.3:0/2172733571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c009a70 con 0x7fe89406aaa0 2024-09-13T16:51:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe891e00700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe89406aaa0 0x7fe894150e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c4d0 tx=0x7fe88400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe88a000700 1 -- 172.21.15.3:0/2172733571 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8840011d0 con 0x7fe89406aaa0 2024-09-13T16:51:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe88a000700 1 -- 172.21.15.3:0/2172733571 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe884005d90 con 0x7fe89406aaa0 2024-09-13T16:51:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe88a000700 1 -- 172.21.15.3:0/2172733571 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8840072d0 con 0x7fe89406aaa0 2024-09-13T16:51:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe894152c20 con 0x7fe89406aaa0 2024-09-13T16:51:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8941531e0 con 0x7fe89406aaa0 2024-09-13T16:51:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.165+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe894159e20 con 0x7fe89406aaa0 2024-09-13T16:51:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.169+0000 7fe88a000700 1 -- 172.21.15.3:0/2172733571 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe884007430 con 0x7fe89406aaa0 2024-09-13T16:51:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.169+0000 7fe88a000700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe874066730 0x7fe874068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.169+0000 7fe88a000700 1 -- 172.21.15.3:0/2172733571 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe8840837f0 con 0x7fe89406aaa0 2024-09-13T16:51:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.169+0000 7fe891400700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe874066730 0x7fe874068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.173+0000 7fe891400700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe874066730 0x7fe874068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe89415a480 tx=0x7fe880009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:09.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.177+0000 7fe88a000700 1 -- 172.21.15.3:0/2172733571 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe884050920 con 0x7fe89406aaa0 2024-09-13T16:51:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.361+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe89406af30 con 0x7fe89406aaa0 2024-09-13T16:51:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.361+0000 7fe88a000700 1 -- 172.21.15.3:0/2172733571 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe8840170a0 con 0x7fe89406aaa0 2024-09-13T16:51:09.365 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.361+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe874066730 msgr2=0x7fe874068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.361+0000 7fe893200700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe874066730 0x7fe874068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe89415a480 tx=0x7fe880009040 comp rx=0 tx=0).stop 2024-09-13T16:51:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.361+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe89406aaa0 msgr2=0x7fe894150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.361+0000 7fe893200700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe89406aaa0 0x7fe894150e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c4d0 tx=0x7fe88400c890 comp rx=0 tx=0).stop 2024-09-13T16:51:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.365+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 shutdown_connections 2024-09-13T16:51:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.365+0000 7fe893200700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe874066730 0x7fe874068bf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.365+0000 7fe893200700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941507b0 0x7fe8940705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.365+0000 7fe893200700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940b2ce0 0x7fe89415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.365+0000 7fe893200700 1 --2- 172.21.15.3:0/2172733571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe89406aaa0 0x7fe894150e90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.365+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 >> 172.21.15.3:0/2172733571 conn(0x7fe8940af620 msgr2=0x7fe89406dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.365+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 shutdown_connections 2024-09-13T16:51:09.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.365+0000 7fe893200700 1 -- 172.21.15.3:0/2172733571 wait complete. 2024-09-13T16:51:09.393 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.565+0000 7fb62e000700 1 Processor -- start 2024-09-13T16:51:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.565+0000 7fb62e000700 1 -- start start 2024-09-13T16:51:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6280b3f40 0x7fb6280b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62e000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb62815d4d0 0x7fb62815d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62e000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6281617e0 con 0x7fb62806f8e0 2024-09-13T16:51:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62e000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62815e2a0 con 0x7fb62815d4d0 2024-09-13T16:51:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62e000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb628160a20 con 0x7fb6280b3f40 2024-09-13T16:51:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62cc00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6280b3f40 0x7fb6280b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62cc00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6280b3f40 0x7fb6280b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:36692/0 (socket says 172.21.15.3:36692) 2024-09-13T16:51:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62cc00700 1 -- 172.21.15.3:0/2857857547 learned_addr learned my addr 172.21.15.3:0/2857857547 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb627e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb62806f8e0 0x7fb62806fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:49044/0 (socket says 172.21.15.3:49044) 2024-09-13T16:51:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62d600700 1 --2- 172.21.15.3:0/2857857547 >> v2:172.21.15.157:3300/0 conn(0x7fb62815d4d0 0x7fb62815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62cc00700 1 -- 172.21.15.3:0/2857857547 >> v2:172.21.15.157:3300/0 conn(0x7fb62815d4d0 msgr2=0x7fb62815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62cc00700 1 --2- 172.21.15.3:0/2857857547 >> v2:172.21.15.157:3300/0 conn(0x7fb62815d4d0 0x7fb62815d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62cc00700 1 -- 172.21.15.3:0/2857857547 >> v1:172.21.15.157:6789/0 conn(0x7fb62806f8e0 legacy=0x7fb62806fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb627400700 1 -- 172.21.15.3:0/2857857547 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909075897 0 0) 0x7fb6281617e0 con 0x7fb62806f8e0 2024-09-13T16:51:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62cc00700 1 -- 172.21.15.3:0/2857857547 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628160d00 con 0x7fb6280b3f40 2024-09-13T16:51:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.569+0000 7fb62cc00700 1 --2- 172.21.15.3:0/2857857547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6280b3f40 0x7fb6280b4320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009e50 tx=0x7fb61c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33546cd38a186796 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb627400700 1 -- 172.21.15.3:0/2857857547 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00e070 con 0x7fb6280b3f40 2024-09-13T16:51:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb627400700 1 -- 172.21.15.3:0/2857857547 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c00b910 con 0x7fb6280b3f40 2024-09-13T16:51:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb627400700 1 -- 172.21.15.3:0/2857857547 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00bcc0 con 0x7fb6280b3f40 2024-09-13T16:51:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb62e000700 1 -- 172.21.15.3:0/2857857547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6280b3f40 msgr2=0x7fb6280b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb62e000700 1 --2- 172.21.15.3:0/2857857547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6280b3f40 0x7fb6280b4320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009e50 tx=0x7fb61c00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb62e000700 1 -- 172.21.15.3:0/2857857547 shutdown_connections 2024-09-13T16:51:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb62e000700 1 --2- 172.21.15.3:0/2857857547 >> v2:172.21.15.157:3300/0 conn(0x7fb62815d4d0 0x7fb62815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb62e000700 1 --2- 172.21.15.3:0/2857857547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6280b3f40 0x7fb6280b4320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb62e000700 1 -- 172.21.15.3:0/2857857547 >> 172.21.15.3:0/2857857547 conn(0x7fb6280b2910 msgr2=0x7fb6280b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb62e000700 1 -- 172.21.15.3:0/2857857547 shutdown_connections 2024-09-13T16:51:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.573+0000 7fb62e000700 1 -- 172.21.15.3:0/2857857547 wait complete. 2024-09-13T16:51:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 Processor -- start 2024-09-13T16:51:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 -- start start 2024-09-13T16:51:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb62806f8e0 0x7fb62813c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6280b3f40 0x7fb6281f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62815d4d0 0x7fb628144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628141020 con 0x7fb62806f8e0 2024-09-13T16:51:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628140410 con 0x7fb6280b3f40 2024-09-13T16:51:09.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281406f0 con 0x7fb62815d4d0 2024-09-13T16:51:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb62806f8e0 0x7fb62813c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb62806f8e0 0x7fb62813c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33056/0 (socket says 172.21.15.3:33056) 2024-09-13T16:51:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 -- 172.21.15.3:0/117704402 learned_addr learned my addr 172.21.15.3:0/117704402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb627e00700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6280b3f40 0x7fb6281f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62d600700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62815d4d0 0x7fb628144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 -- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62815d4d0 msgr2=0x7fb628144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62815d4d0 0x7fb628144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 -- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6280b3f40 msgr2=0x7fb6281f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6280b3f40 0x7fb6281f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 -- 172.21.15.3:0/117704402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c009b30 con 0x7fb62806f8e0 2024-09-13T16:51:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb627e00700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6280b3f40 0x7fb6281f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62cc00700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb62806f8e0 0x7fb62813c380 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b370 tx=0x7fb61c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb624c00700 1 -- 172.21.15.3:0/117704402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0092e0 con 0x7fb62806f8e0 2024-09-13T16:51:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb624c00700 1 -- 172.21.15.3:0/117704402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0099b0 con 0x7fb62806f8e0 2024-09-13T16:51:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6281409d0 con 0x7fb62806f8e0 2024-09-13T16:51:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.577+0000 7fb624c00700 1 -- 172.21.15.3:0/117704402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c01c660 con 0x7fb62806f8e0 2024-09-13T16:51:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.581+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6281f29a0 con 0x7fb62806f8e0 2024-09-13T16:51:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.581+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6280b4320 con 0x7fb62806f8e0 2024-09-13T16:51:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.581+0000 7fb624c00700 1 -- 172.21.15.3:0/117704402 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb61c01c7c0 con 0x7fb62806f8e0 2024-09-13T16:51:09.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.585+0000 7fb624c00700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb608066730 0x7fb608068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:09.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.585+0000 7fb624c00700 1 -- 172.21.15.3:0/117704402 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb61c0861f0 con 0x7fb62806f8e0 2024-09-13T16:51:09.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.585+0000 7fb627e00700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb608066730 0x7fb608068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:09.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.585+0000 7fb627e00700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb608066730 0x7fb608068bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6280709f0 tx=0x7fb614009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:09.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.589+0000 7fb624c00700 1 -- 172.21.15.3:0/117704402 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb61c053260 con 0x7fb62806f8e0 2024-09-13T16:51:09.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.825+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6280b1ff0 con 0x7fb62806f8e0 2024-09-13T16:51:09.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.825+0000 7fb624c00700 1 -- 172.21.15.3:0/117704402 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4727 (secure 0 0 0) 0x7fb61c056880 con 0x7fb62806f8e0 2024-09-13T16:51:09.830 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:09.830 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:55.700655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:51:09.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb608066730 msgr2=0x7fb608068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb608066730 0x7fb608068bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6280709f0 tx=0x7fb614009040 comp rx=0 tx=0).stop 2024-09-13T16:51:09.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb62806f8e0 msgr2=0x7fb62813c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:09.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb62806f8e0 0x7fb62813c380 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b370 tx=0x7fb61c004af0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 shutdown_connections 2024-09-13T16:51:09.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb608066730 0x7fb608068bf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62815d4d0 0x7fb628144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6280b3f40 0x7fb6281f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 --2- 172.21.15.3:0/117704402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb62806f8e0 0x7fb62813c380 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:09.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 >> 172.21.15.3:0/117704402 conn(0x7fb6280b2910 msgr2=0x7fb62815bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:09.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 shutdown_connections 2024-09-13T16:51:09.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:09.829+0000 7fb62e000700 1 -- 172.21.15.3:0/117704402 wait complete. 2024-09-13T16:51:09.836 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-13T16:51:09.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:10.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.025+0000 7fbff3e00700 1 Processor -- start 2024-09-13T16:51:10.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.025+0000 7fbff3e00700 1 -- start start 2024-09-13T16:51:10.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff3e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff41443b0 0x7fbff40b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff3e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbff4144790 0x7fbff406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff3e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff406cf70 con 0x7fbff406cb20 2024-09-13T16:51:10.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff3e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40ce8e0 con 0x7fbff4144790 2024-09-13T16:51:10.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff3e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40ccd30 con 0x7fbff41443b0 2024-09-13T16:51:10.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbfea000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbff4144790 0x7fbff406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbfea000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbff4144790 0x7fbff406c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33068/0 (socket says 172.21.15.3:33068) 2024-09-13T16:51:10.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbfea000700 1 -- 172.21.15.3:0/3029543787 learned_addr learned my addr 172.21.15.3:0/3029543787 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:10.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff2a00700 1 --2- 172.21.15.3:0/3029543787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff41443b0 0x7fbff40b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbfea000700 1 -- 172.21.15.3:0/3029543787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff41443b0 msgr2=0x7fbff40b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:10.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbfea000700 1 --2- 172.21.15.3:0/3029543787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff41443b0 0x7fbff40b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbfea000700 1 -- 172.21.15.3:0/3029543787 >> v1:172.21.15.157:6789/0 conn(0x7fbff406cb20 legacy=0x7fbff406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbfea000700 1 -- 172.21.15.3:0/3029543787 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40cd010 con 0x7fbff4144790 2024-09-13T16:51:10.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbfea000700 1 --2- 172.21.15.3:0/3029543787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff4144790 0x7fbff406c5e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009910 tx=0x7fbfdc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5942a5c9681b6ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff2000700 1 -- 172.21.15.3:0/3029543787 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc011830 con 0x7fbff4144790 2024-09-13T16:51:10.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff2000700 1 -- 172.21.15.3:0/3029543787 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc011990 con 0x7fbff4144790 2024-09-13T16:51:10.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.029+0000 7fbff2000700 1 -- 172.21.15.3:0/3029543787 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc011d40 con 0x7fbff4144790 2024-09-13T16:51:10.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.033+0000 7fbff3e00700 1 -- 172.21.15.3:0/3029543787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff4144790 msgr2=0x7fbff406c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.033+0000 7fbff3e00700 1 --2- 172.21.15.3:0/3029543787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff4144790 0x7fbff406c5e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009910 tx=0x7fbfdc009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:10.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.033+0000 7fbff3e00700 1 -- 172.21.15.3:0/3029543787 shutdown_connections 2024-09-13T16:51:10.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.033+0000 7fbff3e00700 1 --2- 172.21.15.3:0/3029543787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff4144790 0x7fbff406c5e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.033+0000 7fbff3e00700 1 --2- 172.21.15.3:0/3029543787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff41443b0 0x7fbff40b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.033+0000 7fbff3e00700 1 -- 172.21.15.3:0/3029543787 >> 172.21.15.3:0/3029543787 conn(0x7fbff40b0cf0 msgr2=0x7fbff40b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:10.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.033+0000 7fbff3e00700 1 -- 172.21.15.3:0/3029543787 shutdown_connections 2024-09-13T16:51:10.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.033+0000 7fbff3e00700 1 -- 172.21.15.3:0/3029543787 wait complete. 2024-09-13T16:51:10.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3e00700 1 Processor -- start 2024-09-13T16:51:10.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3e00700 1 -- start start 2024-09-13T16:51:10.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff406cb20 0x7fbff40c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff41443b0 0x7fbff4133810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff40c5be0 0x7fbff41f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4134250 con 0x7fbff40c5be0 2024-09-13T16:51:10.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff415ace0 con 0x7fbff41443b0 2024-09-13T16:51:10.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4133d50 con 0x7fbff406cb20 2024-09-13T16:51:10.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff40c5be0 0x7fbff41f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff40c5be0 0x7fbff41f6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33070/0 (socket says 172.21.15.3:33070) 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 -- 172.21.15.3:0/2047962280 learned_addr learned my addr 172.21.15.3:0/2047962280 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff2a00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff406cb20 0x7fbff40c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbfea000700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff41443b0 0x7fbff4133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 -- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff406cb20 msgr2=0x7fbff40c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff406cb20 0x7fbff40c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 -- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff41443b0 msgr2=0x7fbff4133810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff41443b0 0x7fbff4133810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 -- 172.21.15.3:0/2047962280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc009570 con 0x7fbff40c5be0 2024-09-13T16:51:10.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.037+0000 7fbff3400700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff40c5be0 0x7fbff41f6b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c480 tx=0x7fbfec00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.041+0000 7fbfebe00700 1 -- 172.21.15.3:0/2047962280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0052d0 con 0x7fbff40c5be0 2024-09-13T16:51:10.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.041+0000 7fbfebe00700 1 -- 172.21.15.3:0/2047962280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00ce90 con 0x7fbff40c5be0 2024-09-13T16:51:10.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.041+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff41f7090 con 0x7fbff40c5be0 2024-09-13T16:51:10.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.041+0000 7fbfebe00700 1 -- 172.21.15.3:0/2047962280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0036a0 con 0x7fbff40c5be0 2024-09-13T16:51:10.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.041+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff40c63f0 con 0x7fbff40c5be0 2024-09-13T16:51:10.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.041+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff406c5e0 con 0x7fbff40c5be0 2024-09-13T16:51:10.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.045+0000 7fbfebe00700 1 -- 172.21.15.3:0/2047962280 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbfec003800 con 0x7fbff40c5be0 2024-09-13T16:51:10.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.045+0000 7fbfebe00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbfd0066760 0x7fbfd0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.045+0000 7fbfebe00700 1 -- 172.21.15.3:0/2047962280 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbfec0836d0 con 0x7fbff40c5be0 2024-09-13T16:51:10.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.045+0000 7fbff2a00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbfd0066760 0x7fbfd0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.045+0000 7fbff2a00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbfd0066760 0x7fbfd0068c20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40113e0 tx=0x7fbfe400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.053+0000 7fbfebe00700 1 -- 172.21.15.3:0/2047962280 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbfec050630 con 0x7fbff40c5be0 2024-09-13T16:51:10.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.253+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbff406cf70 con 0x7fbff40c5be0 2024-09-13T16:51:10.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbfebe00700 1 -- 172.21.15.3:0/2047962280 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18298 (secure 0 0 0) 0x7fbff406cf70 con 0x7fbff40c5be0 2024-09-13T16:51:10.260 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:10.261 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:50:36.084744+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbfd0066760 msgr2=0x7fbfd0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbfd0066760 0x7fbfd0068c20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40113e0 tx=0x7fbfe400b040 comp rx=0 tx=0).stop 2024-09-13T16:51:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff40c5be0 msgr2=0x7fbff41f6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff40c5be0 0x7fbff41f6b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c480 tx=0x7fbfec00c790 comp rx=0 tx=0).stop 2024-09-13T16:51:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 shutdown_connections 2024-09-13T16:51:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbfd0066760 0x7fbfd0068c20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff40c5be0 0x7fbff41f6b50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff41443b0 0x7fbff4133810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 --2- 172.21.15.3:0/2047962280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff406cb20 0x7fbff40c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 >> 172.21.15.3:0/2047962280 conn(0x7fbff40b0cf0 msgr2=0x7fbff41432d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 shutdown_connections 2024-09-13T16:51:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.257+0000 7fbff3e00700 1 -- 172.21.15.3:0/2047962280 wait complete. 2024-09-13T16:51:10.288 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:10.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.457+0000 7ff67f000700 1 Processor -- start 2024-09-13T16:51:10.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67f000700 1 -- start start 2024-09-13T16:51:10.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67f000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff678144790 0x7ff67806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67f000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff67806cb20 0x7ff67815b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67f000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6780b4100 con 0x7ff6781443b0 2024-09-13T16:51:10.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67f000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67806cfb0 con 0x7ff678144790 2024-09-13T16:51:10.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67f000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6780ce8e0 con 0x7ff67806cb20 2024-09-13T16:51:10.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67d200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff678144790 0x7ff67806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67e600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff67806cb20 0x7ff67815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67d200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff678144790 0x7ff67806c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33072/0 (socket says 172.21.15.3:33072) 2024-09-13T16:51:10.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67dc00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff6781443b0 0x7ff6780b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:49058/0 (socket says 172.21.15.3:49058) 2024-09-13T16:51:10.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.461+0000 7ff67d200700 1 -- 172.21.15.3:0/3721263194 learned_addr learned my addr 172.21.15.3:0/3721263194 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:10.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67d200700 1 -- 172.21.15.3:0/3721263194 >> v2:172.21.15.157:3300/0 conn(0x7ff67806cb20 msgr2=0x7ff67815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:10.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67d200700 1 --2- 172.21.15.3:0/3721263194 >> v2:172.21.15.157:3300/0 conn(0x7ff67806cb20 0x7ff67815b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67d200700 1 -- 172.21.15.3:0/3721263194 >> v1:172.21.15.157:6789/0 conn(0x7ff6781443b0 legacy=0x7ff6780b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67d200700 1 -- 172.21.15.3:0/3721263194 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780ccd30 con 0x7ff678144790 2024-09-13T16:51:10.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67e600700 1 --2- 172.21.15.3:0/3721263194 >> v2:172.21.15.157:3300/0 conn(0x7ff67806cb20 0x7ff67815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:10.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff677e00700 1 -- 172.21.15.3:0/3721263194 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475630176 0 0) 0x7ff6780b4100 con 0x7ff6781443b0 2024-09-13T16:51:10.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67d200700 1 --2- 172.21.15.3:0/3721263194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff678144790 0x7ff67806c5e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6780b3280 tx=0x7ff664009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efa1d43b147058e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff677e00700 1 -- 172.21.15.3:0/3721263194 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664011830 con 0x7ff678144790 2024-09-13T16:51:10.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff677e00700 1 -- 172.21.15.3:0/3721263194 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff664011990 con 0x7ff678144790 2024-09-13T16:51:10.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff677e00700 1 -- 172.21.15.3:0/3721263194 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664011d20 con 0x7ff678144790 2024-09-13T16:51:10.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67f000700 1 -- 172.21.15.3:0/3721263194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff678144790 msgr2=0x7ff67806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67f000700 1 --2- 172.21.15.3:0/3721263194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff678144790 0x7ff67806c5e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6780b3280 tx=0x7ff664009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67f000700 1 -- 172.21.15.3:0/3721263194 shutdown_connections 2024-09-13T16:51:10.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67f000700 1 --2- 172.21.15.3:0/3721263194 >> v2:172.21.15.157:3300/0 conn(0x7ff67806cb20 0x7ff67815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67f000700 1 --2- 172.21.15.3:0/3721263194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff678144790 0x7ff67806c5e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67f000700 1 -- 172.21.15.3:0/3721263194 >> 172.21.15.3:0/3721263194 conn(0x7ff6780b0cf0 msgr2=0x7ff6780b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:10.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.465+0000 7ff67f000700 1 -- 172.21.15.3:0/3721263194 shutdown_connections 2024-09-13T16:51:10.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 -- 172.21.15.3:0/3721263194 wait complete. 2024-09-13T16:51:10.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 Processor -- start 2024-09-13T16:51:10.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 -- start start 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff67806cb20 0x7ff678159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6781443b0 0x7ff67806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6780703e0 0x7ff6781f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6780ae1a0 con 0x7ff6781443b0 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff678070840 con 0x7ff6780703e0 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67f000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6780ad100 con 0x7ff67806cb20 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67d200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6781443b0 0x7ff67806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67d200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6781443b0 0x7ff67806fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33084/0 (socket says 172.21.15.3:33084) 2024-09-13T16:51:10.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.469+0000 7ff67d200700 1 -- 172.21.15.3:0/3404198133 learned_addr learned my addr 172.21.15.3:0/3404198133 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67e600700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6780703e0 0x7ff6781f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67dc00700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff67806cb20 0x7ff678159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67d200700 1 -- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff67806cb20 msgr2=0x7ff678159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67d200700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff67806cb20 0x7ff678159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67d200700 1 -- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6780703e0 msgr2=0x7ff6781f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67d200700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6780703e0 0x7ff6781f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67d200700 1 -- 172.21.15.3:0/3404198133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670009a70 con 0x7ff6781443b0 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67dc00700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff67806cb20 0x7ff678159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67d200700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6781443b0 0x7ff67806fea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6780cc8f0 tx=0x7ff664004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff675600700 1 -- 172.21.15.3:0/3404198133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640045d0 con 0x7ff6781443b0 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff675600700 1 -- 172.21.15.3:0/3404198133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66401f070 con 0x7ff6781443b0 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff675600700 1 -- 172.21.15.3:0/3404198133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640038e0 con 0x7ff6781443b0 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff664009570 con 0x7ff6781443b0 2024-09-13T16:51:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6780ad6c0 con 0x7ff6781443b0 2024-09-13T16:51:10.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.473+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6780cca20 con 0x7ff6781443b0 2024-09-13T16:51:10.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.477+0000 7ff675600700 1 -- 172.21.15.3:0/3404198133 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff664003a40 con 0x7ff6781443b0 2024-09-13T16:51:10.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.477+0000 7ff675600700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff658066730 0x7ff658068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.477+0000 7ff675600700 1 -- 172.21.15.3:0/3404198133 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff664021070 con 0x7ff6781443b0 2024-09-13T16:51:10.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.477+0000 7ff67dc00700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff658066730 0x7ff658068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.481+0000 7ff67dc00700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff658066730 0x7ff658068bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff66c000f60 tx=0x7ff66c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.485+0000 7ff675600700 1 -- 172.21.15.3:0/3404198133 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff664052b10 con 0x7ff6781443b0 2024-09-13T16:51:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.725+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6780b3d30 con 0x7ff6781443b0 2024-09-13T16:51:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff675600700 1 -- 172.21.15.3:0/3404198133 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4727 (secure 0 0 0) 0x7ff664056130 con 0x7ff6781443b0 2024-09-13T16:51:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:10.732 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:55.700655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff658066730 msgr2=0x7ff658068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff658066730 0x7ff658068bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff66c000f60 tx=0x7ff66c009040 comp rx=0 tx=0).stop 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6781443b0 msgr2=0x7ff67806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6781443b0 0x7ff67806fea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6780cc8f0 tx=0x7ff664004770 comp rx=0 tx=0).stop 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 shutdown_connections 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff658066730 0x7ff658068bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6780703e0 0x7ff6781f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6781443b0 0x7ff67806fea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 --2- 172.21.15.3:0/3404198133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff67806cb20 0x7ff678159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 >> 172.21.15.3:0/3404198133 conn(0x7ff6780b0cf0 msgr2=0x7ff678140cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 shutdown_connections 2024-09-13T16:51:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:10.729+0000 7ff67f000700 1 -- 172.21.15.3:0/3404198133 wait complete. 2024-09-13T16:51:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-13T16:51:11.159 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:50:30.569564+0000', 'modified': '2024-09-13T16:50:55.700655+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6837/3884968743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 3884968743}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 3884968743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:51:11.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:51:11.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-13T16:51:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.337+0000 7fbd27200700 1 Processor -- start 2024-09-13T16:51:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.337+0000 7fbd27200700 1 -- start start 2024-09-13T16:51:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.337+0000 7fbd27200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.337+0000 7fbd27200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd200cfbb0 con 0x7fbd200b34b0 2024-09-13T16:51:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.337+0000 7fbd27200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd2015b320 con 0x7fbd200b5570 2024-09-13T16:51:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.337+0000 7fbd27200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2015be80 con 0x7fbd2015aee0 2024-09-13T16:51:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd26800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbd200b34b0 0x7fbd2006a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34674/0 (socket says 172.21.15.3:34674) 2024-09-13T16:51:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd25e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbd200b5570 0x7fbd200b3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:49060/0 (socket says 172.21.15.3:49060) 2024-09-13T16:51:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd26800700 1 -- 172.21.15.3:0/3940956004 learned_addr learned my addr 172.21.15.3:0/3940956004 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd25400700 1 --2- 172.21.15.3:0/3940956004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd24a00700 1 -- 172.21.15.3:0/3940956004 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506466680 0 0) 0x7fbd2015b320 con 0x7fbd200b5570 2024-09-13T16:51:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd24a00700 1 -- 172.21.15.3:0/3940956004 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd080035d0 con 0x7fbd200b5570 2024-09-13T16:51:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd24a00700 1 -- 172.21.15.3:0/3940956004 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705553752 0 0) 0x7fbd200cfbb0 con 0x7fbd200b34b0 2024-09-13T16:51:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd25400700 1 -- 172.21.15.3:0/3940956004 >> v1:172.21.15.157:6789/0 conn(0x7fbd200b5570 legacy=0x7fbd200b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd25400700 1 -- 172.21.15.3:0/3940956004 >> v1:172.21.15.3:6789/0 conn(0x7fbd200b34b0 legacy=0x7fbd2006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd25400700 1 -- 172.21.15.3:0/3940956004 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2015b5e0 con 0x7fbd2015aee0 2024-09-13T16:51:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd25400700 1 --2- 172.21.15.3:0/3940956004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200cf670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009880 tx=0x7fbd10009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad5714612a9c4608 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd24a00700 1 -- 172.21.15.3:0/3940956004 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10011830 con 0x7fbd2015aee0 2024-09-13T16:51:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd24a00700 1 -- 172.21.15.3:0/3940956004 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd10011990 con 0x7fbd2015aee0 2024-09-13T16:51:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.341+0000 7fbd24a00700 1 -- 172.21.15.3:0/3940956004 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10011d20 con 0x7fbd2015aee0 2024-09-13T16:51:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 -- 172.21.15.3:0/3940956004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 msgr2=0x7fbd200cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 --2- 172.21.15.3:0/3940956004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200cf670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009880 tx=0x7fbd10009b90 comp rx=0 tx=0).stop 2024-09-13T16:51:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 -- 172.21.15.3:0/3940956004 shutdown_connections 2024-09-13T16:51:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 --2- 172.21.15.3:0/3940956004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200cf670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 -- 172.21.15.3:0/3940956004 >> 172.21.15.3:0/3940956004 conn(0x7fbd20071dc0 msgr2=0x7fbd2006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 -- 172.21.15.3:0/3940956004 shutdown_connections 2024-09-13T16:51:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 -- 172.21.15.3:0/3940956004 wait complete. 2024-09-13T16:51:11.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 Processor -- start 2024-09-13T16:51:11.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.345+0000 7fbd27200700 1 -- start start 2024-09-13T16:51:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd27200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd200b34b0 0x7fbd201478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd27200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd200b5570 0x7fbd20155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd27200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd27200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2014df30 con 0x7fbd200b5570 2024-09-13T16:51:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd27200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20147e00 con 0x7fbd200b34b0 2024-09-13T16:51:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd27200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd201494d0 con 0x7fbd2015aee0 2024-09-13T16:51:11.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd200b5570 0x7fbd20155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd200b5570 0x7fbd20155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33088/0 (socket says 172.21.15.3:33088) 2024-09-13T16:51:11.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 -- 172.21.15.3:0/293022798 learned_addr learned my addr 172.21.15.3:0/293022798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25e00700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd200b34b0 0x7fbd201478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd26800700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 -- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 msgr2=0x7fbd200c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 -- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd200b34b0 msgr2=0x7fbd201478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd200b34b0 0x7fbd201478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 -- 172.21.15.3:0/293022798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100094b0 con 0x7fbd200b5570 2024-09-13T16:51:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd25400700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd200b5570 0x7fbd20155bc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd10013040 tx=0x7fbd10004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd1a000700 1 -- 172.21.15.3:0/293022798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100198d0 con 0x7fbd200b5570 2024-09-13T16:51:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd1a000700 1 -- 172.21.15.3:0/293022798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1001e070 con 0x7fbd200b5570 2024-09-13T16:51:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd1a000700 1 -- 172.21.15.3:0/293022798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10019c50 con 0x7fbd200b5570 2024-09-13T16:51:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd27200700 1 -- 172.21.15.3:0/293022798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd20149790 con 0x7fbd200b5570 2024-09-13T16:51:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.349+0000 7fbd27200700 1 -- 172.21.15.3:0/293022798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2014d870 con 0x7fbd200b5570 2024-09-13T16:51:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.353+0000 7fbd1a000700 1 -- 172.21.15.3:0/293022798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbd1002d450 con 0x7fbd200b5570 2024-09-13T16:51:11.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.353+0000 7fbd1a000700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbd04066730 0x7fbd04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.357+0000 7fbd1a000700 1 -- 172.21.15.3:0/293022798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd10085330 con 0x7fbd200b5570 2024-09-13T16:51:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.357+0000 7fbd25e00700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbd04066730 0x7fbd04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.357+0000 7fbd25e00700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbd04066730 0x7fbd04068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007fd0 tx=0x7fbd14007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.357+0000 7fbd2bb70740 1 Processor -- start 2024-09-13T16:51:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.357+0000 7fbd2bb70740 1 -- start start 2024-09-13T16:51:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2304fda0 0x23050180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 0x23018890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23018dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f77d80 con 0x230506c0 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f75510 con 0x2302ed00 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x22f70e00 con 0x2304fda0 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23018dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23018dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33100/0 (socket says 172.21.15.3:33100) 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 -- 172.21.15.3:0/2288029572 learned_addr learned my addr 172.21.15.3:0/2288029572 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd25400700 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 0x23018890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd25e00700 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2304fda0 0x23050180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 -- 172.21.15.3:0/2288029572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2304fda0 msgr2=0x23050180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2304fda0 0x23050180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 -- 172.21.15.3:0/2288029572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 msgr2=0x23018890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 0x23018890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 -- 172.21.15.3:0/2288029572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22de3960 con 0x230506c0 2024-09-13T16:51:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd26800700 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23018dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0070e0 tx=0x7fbd1c0073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a60a078f226c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd0e600700 1 -- 172.21.15.3:0/2288029572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0133f0 con 0x230506c0 2024-09-13T16:51:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd0e600700 1 -- 172.21.15.3:0/2288029572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c013550 con 0x230506c0 2024-09-13T16:51:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd0e600700 1 -- 172.21.15.3:0/2288029572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c013900 con 0x230506c0 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 -- 172.21.15.3:0/2288029572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 msgr2=0x23018dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23018dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0070e0 tx=0x7fbd1c0073f0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 -- 172.21.15.3:0/2288029572 shutdown_connections 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23018dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 0x23018890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/2288029572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2304fda0 0x23050180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 -- 172.21.15.3:0/2288029572 >> 172.21.15.3:0/2288029572 conn(0x22d984d0 msgr2=0x23031c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 -- 172.21.15.3:0/2288029572 shutdown_connections 2024-09-13T16:51:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.361+0000 7fbd2bb70740 1 -- 172.21.15.3:0/2288029572 wait complete. 2024-09-13T16:51:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 1 -- start start 2024-09-13T16:51:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 1 Processor -- start 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x23008390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x23050aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23050fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f2e730 con 0x230506c0 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f345b0 con 0x2304fda0 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x22e1caa0 con 0x2302ed00 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd2bb70740 10 client.0 _ll_register_callbacks cb 0x22f6a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23050fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23050fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33102/0 (socket says 172.21.15.3:33102) 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 -- 172.21.15.3:0/430096674 learned_addr learned my addr 172.21.15.3:0/430096674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd25400700 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x23050aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd25e00700 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x23008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 -- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 msgr2=0x23008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:11.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x23008390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 -- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 msgr2=0x23050aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:11.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x23050aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 -- 172.21.15.3:0/430096674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1c001f80 con 0x230506c0 2024-09-13T16:51:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.365+0000 7fbd26800700 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23050fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c007740 tx=0x7fbd1c0019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708a11cfaf081029 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd02a00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd02a00700 1 -- 172.21.15.3:0/430096674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0133f0 con 0x230506c0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd02a00700 1 -- 172.21.15.3:0/430096674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd1c01d810 con 0x230506c0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 1 -- 172.21.15.3:0/430096674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22da51e0 con 0x230506c0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd02a00700 1 -- 172.21.15.3:0/430096674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd1c001710 con 0x230506c0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 10 client.4878 fetch_fsmap learned FSMap version 8 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 1 -- 172.21.15.3:0/430096674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x22f37220 con 0x230506c0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd02a00700 1 -- 172.21.15.3:0/430096674 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22da51e0 con 0x230506c0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd02a00700 1 -- 172.21.15.3:0/430096674 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fbd1c027d20 con 0x230506c0 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 10 client.4878 fetch_fsmap finished waiting for FSMap version 8 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 10 client.4878 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-13T16:51:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 20 client.4878 populate_metadata read hostname 'smithi003' 2024-09-13T16:51:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x230061e0 0x230065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 4 client.4878 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:51:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd2bb70740 1 -- 172.21.15.3:0/430096674 --> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x22def460 con 0x230061e0 2024-09-13T16:51:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd25e00700 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x230061e0 0x230065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd25e00700 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x230061e0 0x230065c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff77ae1616c52b8 server_cookie=41eef910588b8cb in_seq=0 out_seq=0 2024-09-13T16:51:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.369+0000 7fbd02a00700 10 client.4878 ms_handle_connect on v2:172.21.15.157:6834/3884968743 2024-09-13T16:51:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd02a00700 1 -- 172.21.15.3:0/430096674 <== mds.0 v2:172.21.15.157:6834/3884968743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x22def460 con 0x230061e0 2024-09-13T16:51:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd02a00700 10 client.4878 handle_command_reply: tid=0 2024-09-13T16:51:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 client.4878 shutdown 2024-09-13T16:51:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 -- 172.21.15.3:0/430096674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 msgr2=0x23050fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23050fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c007740 tx=0x7fbd1c0019d0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 -- 172.21.15.3:0/430096674 shutdown_connections 2024-09-13T16:51:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x230061e0 0x230065c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x23050fe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x23050aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/430096674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x23008390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd02a00700 0 client.4878 ms_handle_reset on v2:172.21.15.157:6834/3884968743 2024-09-13T16:51:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 -- 172.21.15.3:0/430096674 >> 172.21.15.3:0/430096674 conn(0x22d984d0 msgr2=0x2302b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 -- 172.21.15.3:0/430096674 shutdown_connections 2024-09-13T16:51:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 1 -- 172.21.15.3:0/430096674 wait complete. 2024-09-13T16:51:11.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.373+0000 7fbd2bb70740 20 client.4878 trim_cache size 0 max 16384 2024-09-13T16:51:11.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.393+0000 7fbd2bb70740 1 Processor -- start 2024-09-13T16:51:11.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.393+0000 7fbd2bb70740 1 -- start start 2024-09-13T16:51:11.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.393+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 0x23062ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.393+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2304fda0 0x23063420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.393+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x230506c0 0x23063960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.393+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f3a8d0 con 0x2304fda0 2024-09-13T16:51:11.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.393+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f2d700 con 0x2302ed00 2024-09-13T16:51:11.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.393+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x22fc7580 con 0x230506c0 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2304fda0 0x23063420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2304fda0 0x23063420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33104/0 (socket says 172.21.15.3:33104) 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 -- 172.21.15.3:0/3004652265 learned_addr learned my addr 172.21.15.3:0/3004652265 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd26800700 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x230506c0 0x23063960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25e00700 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 0x23062ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 -- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x230506c0 msgr2=0x23063960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x230506c0 0x23063960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 -- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 msgr2=0x23062ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 0x23062ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 -- 172.21.15.3:0/3004652265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22f1d450 con 0x2304fda0 2024-09-13T16:51:11.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd26800700 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x230506c0 0x23063960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:11.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd25400700 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2304fda0 0x23063420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd100525c0 tx=0x7fbd1005c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683ad7ffbd8be183 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd0f000700 1 -- 172.21.15.3:0/3004652265 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10069ea0 con 0x2304fda0 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd0f000700 1 -- 172.21.15.3:0/3004652265 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd10056110 con 0x2304fda0 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd0f000700 1 -- 172.21.15.3:0/3004652265 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10059e90 con 0x2304fda0 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3004652265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2304fda0 msgr2=0x23063420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2304fda0 0x23063420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd100525c0 tx=0x7fbd1005c040 comp rx=0 tx=0).stop 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3004652265 shutdown_connections 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x230506c0 0x23063960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2304fda0 0x23063420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3004652265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2302ed00 0x23062ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3004652265 >> 172.21.15.3:0/3004652265 conn(0x22d984d0 msgr2=0x23006de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3004652265 shutdown_connections 2024-09-13T16:51:11.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3004652265 wait complete. 2024-09-13T16:51:11.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 -- start start 2024-09-13T16:51:11.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.397+0000 7fbd2bb70740 1 Processor -- start 2024-09-13T16:51:11.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x230b9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x230ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x230ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x23076c40 con 0x230506c0 2024-09-13T16:51:11.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x230513d0 con 0x2304fda0 2024-09-13T16:51:11.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x2300acc0 con 0x2302ed00 2024-09-13T16:51:11.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 10 client.0 _ll_register_callbacks cb 0x22fc7710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:51:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd26800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x230ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x230ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd26800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x230ba8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33106/0 (socket says 172.21.15.3:33106) 2024-09-13T16:51:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd26800700 1 -- 172.21.15.3:0/3151867794 learned_addr learned my addr 172.21.15.3:0/3151867794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25e00700 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x230b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25400700 1 -- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 msgr2=0x230b9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25400700 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x230b9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25400700 1 -- 172.21.15.3:0/3151867794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 msgr2=0x230ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25400700 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x230ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25400700 1 -- 172.21.15.3:0/3151867794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2014d870 con 0x2304fda0 2024-09-13T16:51:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25e00700 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x230b9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd26800700 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x230ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd25400700 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x230ba380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd10025000 tx=0x7fbd10059000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c7444e838fab3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd02a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-13T16:51:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd02a00700 1 -- 172.21.15.3:0/3151867794 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10083280 con 0x2304fda0 2024-09-13T16:51:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd02a00700 1 -- 172.21.15.3:0/3151867794 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd1005e450 con 0x2304fda0 2024-09-13T16:51:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3151867794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd1c013550 con 0x2304fda0 2024-09-13T16:51:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd02a00700 1 -- 172.21.15.3:0/3151867794 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd10059e90 con 0x2304fda0 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 10 client.4885 fetch_fsmap learned FSMap version 8 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.401+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3151867794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x22f11720 con 0x2304fda0 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd02a00700 1 -- 172.21.15.3:0/3151867794 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c013550 con 0x2304fda0 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd02a00700 1 -- 172.21.15.3:0/3151867794 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fbd10056370 con 0x2304fda0 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 10 client.4885 fetch_fsmap finished waiting for FSMap version 8 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 10 client.4885 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 20 client.4885 populate_metadata read hostname 'smithi003' 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x230bb370 0x23066740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 4 client.4885 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3151867794 --> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] -- command(tid 0: {"prefix": "status"}) v1 -- 0x22e141e0 con 0x230bb370 2024-09-13T16:51:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd25e00700 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x230bb370 0x23066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd25e00700 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x230bb370 0x23066740 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07bd66522850e89 server_cookie=7a05f8c9d40b0dcd in_seq=0 out_seq=0 2024-09-13T16:51:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd02a00700 10 client.4885 ms_handle_connect on v2:172.21.15.157:6834/3884968743 2024-09-13T16:51:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd02a00700 1 -- 172.21.15.3:0/3151867794 <== mds.0 v2:172.21.15.157:6834/3884968743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x22e141e0 con 0x230bb370 2024-09-13T16:51:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd02a00700 10 client.4885 handle_command_reply: tid=0 2024-09-13T16:51:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 client.4885 shutdown 2024-09-13T16:51:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 msgr2=0x230ba380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x230ba380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd10025000 tx=0x7fbd10059000 comp rx=0 tx=0).stop 2024-09-13T16:51:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3151867794 shutdown_connections 2024-09-13T16:51:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:6834/3884968743,v1:172.21.15.157:6837/3884968743] conn(0x230bb370 0x23066740 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x230506c0 0x230ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x2304fda0 0x230ba380 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 --2- 172.21.15.3:0/3151867794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x2302ed00 0x230b9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd02a00700 0 client.4885 ms_handle_reset on v2:172.21.15.157:6834/3884968743 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3151867794 >> 172.21.15.3:0/3151867794 conn(0x22d984d0 msgr2=0x23004700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3151867794 shutdown_connections 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 1 -- 172.21.15.3:0/3151867794 wait complete. 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.405+0000 7fbd2bb70740 20 client.4885 trim_cache size 0 max 16384 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:51:11.412 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "61456116-a42d-4065-9bf5-a1daf0be594e", 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4284, 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 40.833692499000001, 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 8, 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 25, 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 45.475361057999997 2024-09-13T16:51:11.413 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:51:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 -- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbd04066730 msgr2=0x7fbd04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbd04066730 0x7fbd04068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007fd0 tx=0x7fbd14007de0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 -- 172.21.15.3:0/293022798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd200b5570 msgr2=0x7fbd20155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd200b5570 0x7fbd20155bc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd10013040 tx=0x7fbd10004770 comp rx=0 tx=0).stop 2024-09-13T16:51:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 -- 172.21.15.3:0/293022798 shutdown_connections 2024-09-13T16:51:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbd04066730 0x7fbd04068bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd2015aee0 0x7fbd200c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd200b5570 0x7fbd20155bc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 --2- 172.21.15.3:0/293022798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd200b34b0 0x7fbd201478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 -- 172.21.15.3:0/293022798 >> 172.21.15.3:0/293022798 conn(0x7fbd20071dc0 msgr2=0x7fbd200b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 -- 172.21.15.3:0/293022798 shutdown_connections 2024-09-13T16:51:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:11.409+0000 7fbd27200700 1 -- 172.21.15.3:0/293022798 wait complete. 2024-09-13T16:51:11.439 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T16:51:11.451 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T16:51:11.451 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T16:51:11.452 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T16:51:11.452 INFO:teuthology.run_tasks:Running task kclient... 2024-09-13T16:51:11.477 INFO:tasks.kclient:Mounting kernel clients... 2024-09-13T16:51:11.477 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-13T16:51:11.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:11.478 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:51:11.499 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-13T16:51:11.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:11.501 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T16:51:11.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:11.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-13T16:51:12.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:12.185 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:51:12.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:12.206 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-13T16:51:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-13T16:51:12.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:51:12.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:12.292 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-13T16:51:12.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:12.312 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-09-13T16:51:12.396 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-09-13T16:51:12.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:51:12.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:12.401 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-13T16:51:12.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:12.461 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-09-13T16:51:12.546 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-09-13T16:51:12.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:51:12.550 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-13T16:51:12.550 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:51:12.550 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T16:51:12.551 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:51:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.721+0000 7f2d0dc00700 1 Processor -- start 2024-09-13T16:51:12.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.725+0000 7f2d0dc00700 1 -- start start 2024-09-13T16:51:12.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.725+0000 7f2d0dc00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d08070db0 0x7f2d08071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:12.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.725+0000 7f2d0dc00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d080b3450 con 0x7f2d0806c540 2024-09-13T16:51:12.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.725+0000 7f2d0dc00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d080b3730 con 0x7f2d0815b660 2024-09-13T16:51:12.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.725+0000 7f2d0dc00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080b40d0 con 0x7f2d08070db0 2024-09-13T16:51:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.725+0000 7f2d07400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2d0806c540 0x7f2d0806c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34684/0 (socket says 172.21.15.3:34684) 2024-09-13T16:51:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.725+0000 7f2d07400700 1 -- 172.21.15.3:0/3911072514 learned_addr learned my addr 172.21.15.3:0/3911072514 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.725+0000 7f2cffe00700 1 --2- 172.21.15.3:0/3911072514 >> v2:172.21.15.157:3300/0 conn(0x7f2d08070db0 0x7f2d08071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:12.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d06000700 1 -- 172.21.15.3:0/3911072514 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816420973 0 0) 0x7f2d080b3450 con 0x7f2d0806c540 2024-09-13T16:51:12.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d06000700 1 -- 172.21.15.3:0/3911072514 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cec0035d0 con 0x7f2d0806c540 2024-09-13T16:51:12.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2cffe00700 1 -- 172.21.15.3:0/3911072514 >> v1:172.21.15.157:6789/0 conn(0x7f2d0815b660 legacy=0x7f2d080b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:12.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2cffe00700 1 -- 172.21.15.3:0/3911072514 >> v1:172.21.15.3:6789/0 conn(0x7f2d0806c540 legacy=0x7f2d0806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:12.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2cffe00700 1 -- 172.21.15.3:0/3911072514 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080b3a10 con 0x7f2d08070db0 2024-09-13T16:51:12.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2cffe00700 1 --2- 172.21.15.3:0/3911072514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 0x7f2d08071240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009850 tx=0x7f2cf0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f25682ae53ce26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:12.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d06000700 1 -- 172.21.15.3:0/3911072514 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0011830 con 0x7f2d08070db0 2024-09-13T16:51:12.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d06000700 1 -- 172.21.15.3:0/3911072514 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf0011990 con 0x7f2d08070db0 2024-09-13T16:51:12.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d06000700 1 -- 172.21.15.3:0/3911072514 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0011d40 con 0x7f2d08070db0 2024-09-13T16:51:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d0dc00700 1 -- 172.21.15.3:0/3911072514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 msgr2=0x7f2d08071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d0dc00700 1 --2- 172.21.15.3:0/3911072514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 0x7f2d08071240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009850 tx=0x7f2cf0009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d0dc00700 1 -- 172.21.15.3:0/3911072514 shutdown_connections 2024-09-13T16:51:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d0dc00700 1 --2- 172.21.15.3:0/3911072514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 0x7f2d08071240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d0dc00700 1 -- 172.21.15.3:0/3911072514 >> 172.21.15.3:0/3911072514 conn(0x7f2d0806a390 msgr2=0x7f2d0806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d0dc00700 1 -- 172.21.15.3:0/3911072514 shutdown_connections 2024-09-13T16:51:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.729+0000 7f2d0dc00700 1 -- 172.21.15.3:0/3911072514 wait complete. 2024-09-13T16:51:12.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.733+0000 7f2d0dc00700 1 Processor -- start 2024-09-13T16:51:12.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.733+0000 7f2d0dc00700 1 -- start start 2024-09-13T16:51:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d0dc00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0806c540 0x7f2d08164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d0dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 0x7f2d08161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d0dc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0815b660 0x7f2d081f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d0dc00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d081639b0 con 0x7f2d0815b660 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d0dc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08163c90 con 0x7f2d08070db0 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d0dc00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08163f70 con 0x7f2d0806c540 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0815b660 0x7f2d081f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0815b660 0x7f2d081f6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33110/0 (socket says 172.21.15.3:33110) 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 -- 172.21.15.3:0/1722910330 learned_addr learned my addr 172.21.15.3:0/1722910330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2cffe00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 0x7f2d08161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d06a00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0806c540 0x7f2d08164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 -- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0806c540 msgr2=0x7f2d08164ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0806c540 0x7f2d08164ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 -- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 msgr2=0x7f2d08161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 0x7f2d08161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 -- 172.21.15.3:0/1722910330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00094b0 con 0x7f2d0815b660 2024-09-13T16:51:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d07400700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0815b660 0x7f2d081f6d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000caa0 tx=0x7f2d0000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2cfea00700 1 -- 172.21.15.3:0/1722910330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00007480 con 0x7f2d0815b660 2024-09-13T16:51:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2cfea00700 1 -- 172.21.15.3:0/1722910330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d000075e0 con 0x7f2d0815b660 2024-09-13T16:51:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d08164250 con 0x7f2d0815b660 2024-09-13T16:51:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2cfea00700 1 -- 172.21.15.3:0/1722910330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00007970 con 0x7f2d0815b660 2024-09-13T16:51:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.737+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d08165530 con 0x7f2d0815b660 2024-09-13T16:51:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.741+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d08164770 con 0x7f2d0815b660 2024-09-13T16:51:12.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.741+0000 7f2cfea00700 1 -- 172.21.15.3:0/1722910330 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d0001b3b0 con 0x7f2d0815b660 2024-09-13T16:51:12.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.745+0000 7f2cfea00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ce4066730 0x7f2ce4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:12.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.745+0000 7f2cfea00700 1 -- 172.21.15.3:0/1722910330 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2d00083d30 con 0x7f2d0815b660 2024-09-13T16:51:12.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.745+0000 7f2d06a00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ce4066730 0x7f2ce4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:12.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.745+0000 7f2d06a00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ce4066730 0x7f2ce4068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8000f60 tx=0x7f2cf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.749+0000 7f2cfea00700 1 -- 172.21.15.3:0/1722910330 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2d00050da0 con 0x7f2d0815b660 2024-09-13T16:51:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.989+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2d0815ba40 con 0x7f2d0815b660 2024-09-13T16:51:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.989+0000 7f2cfea00700 1 -- 172.21.15.3:0/1722910330 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2d000543c0 con 0x7f2d0815b660 2024-09-13T16:51:12.994 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T16:51:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ce4066730 msgr2=0x7f2ce4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ce4066730 0x7f2ce4068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8000f60 tx=0x7f2cf8009040 comp rx=0 tx=0).stop 2024-09-13T16:51:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0815b660 msgr2=0x7f2d081f6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0815b660 0x7f2d081f6d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000caa0 tx=0x7f2d0000cad0 comp rx=0 tx=0).stop 2024-09-13T16:51:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 shutdown_connections 2024-09-13T16:51:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ce4066730 0x7f2ce4068bf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0815b660 0x7f2d081f6d10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d08070db0 0x7f2d08161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 --2- 172.21.15.3:0/1722910330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0806c540 0x7f2d08164ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 >> 172.21.15.3:0/1722910330 conn(0x7f2d0806a390 msgr2=0x7f2d080cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 shutdown_connections 2024-09-13T16:51:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:12.993+0000 7f2d0dc00700 1 -- 172.21.15.3:0/1722910330 wait complete. 2024-09-13T16:51:13.021 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T16:51:13.021 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:51:13.021 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-13T16:51:13.021 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T16:51:13.021 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T16:51:13.021 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T16:51:13.022 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:51:13.022 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:51:13.022 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:51:13.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.022 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-13T16:51:13.039 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:51:13.039 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:51:13.039 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:51:13.039 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:13.039 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-13T16:51:13.039 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:13.039 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:13.039 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42392sec preferred_lft 42392sec 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:13.040 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:13.041 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:13.041 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-13T16:51:13.041 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:51:13.041 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:4d:e8:d2:67 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:13.041 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:51:13.041 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:13.042 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T16:51:13.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.042 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:13.042 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:13.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-09-13T16:51:13.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-09-13T16:51:13.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-09-13T16:51:13.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T16:51:13.042 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:13.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.187 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:51:13.217 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-13T16:51:13.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.221 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-13T16:51:13.284 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-13T16:51:13.284 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:51:13.284 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T16:51:13.284 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:51:13.284 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T16:51:13.285 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:51:13.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.285 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:13.285 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:13.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T16:51:13.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T16:51:13.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T16:51:13.286 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:13.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.417 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:51:13.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.436 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-13T16:51:13.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.502 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:13.502 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:13.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T16:51:13.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T16:51:13.502 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:13.614 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T16:51:13.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.614 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:13.614 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:13.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T16:51:13.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T16:51:13.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T16:51:13.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T16:51:13.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T16:51:13.615 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:13.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.759 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:13.759 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:13.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-13T16:51:13.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T16:51:13.759 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:13.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:51:13.827 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T16:51:13.841 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:51:13.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:51:13.842 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:51:13.901 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-13T16:51:13.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:13.901 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:14.041 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:14.041 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:51:14.788 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:14.788 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:51:14.788 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-13T16:51:14.788 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 0@61456116-a42d-4065-9bf5-a1daf0be594e.cephfs=/ 2024-09-13T16:51:14.788 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.3:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-13T16:51:14.796 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T16:51:14.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:14.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:51:14.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:14.824 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:14.892 INFO:teuthology.orchestra.run.smithi003.stdout:client4664 2024-09-13T16:51:14.893 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:51:14.893 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:14.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:14.955 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:15.027 INFO:teuthology.orchestra.run.smithi003.stdout:client4664 2024-09-13T16:51:15.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:15.027 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:15.090 INFO:teuthology.orchestra.run.smithi003.stdout:client4664 2024-09-13T16:51:15.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:15.091 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/61456116-a42d-4065-9bf5-a1daf0be594e.client4664/status') 2024-09-13T16:51:15.172 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4664 (3)172.21.15.3:0/4217159702 2024-09-13T16:51:15.172 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-09-13T16:51:15.172 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-09-13T16:51:15.172 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-09-13T16:51:15.172 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 0.000169478 s, 407 kB/s 2024-09-13T16:51:15.175 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-13T16:51:15.175 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:51:15.175 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-13T16:51:15.175 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:51:15.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8e04e00700 1 Processor -- start 2024-09-13T16:51:15.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8e04e00700 1 -- start start 2024-09-13T16:51:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8e04e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0006ccf0 0x7f8e001472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8e04e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00070310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8e04e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e00070850 con 0x7f8e00146ee0 2024-09-13T16:51:15.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8e04e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e00070a80 con 0x7f8e0006ccf0 2024-09-13T16:51:15.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8e04e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001626e0 con 0x7f8e0006fea0 2024-09-13T16:51:15.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8dfea00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0006ccf0 0x7f8e001472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8dfea00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0006ccf0 0x7f8e001472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41334/0 (socket says 172.21.15.3:41334) 2024-09-13T16:51:15.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.401+0000 7f8dfea00700 1 -- 172.21.15.3:0/2021722415 learned_addr learned my addr 172.21.15.3:0/2021722415 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:15.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dffe00700 1 --2- 172.21.15.3:0/2021722415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dfea00700 1 -- 172.21.15.3:0/2021722415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 msgr2=0x7f8e00070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:15.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dfea00700 1 --2- 172.21.15.3:0/2021722415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dfea00700 1 -- 172.21.15.3:0/2021722415 >> v1:172.21.15.157:6789/0 conn(0x7f8e00146ee0 legacy=0x7f8e0006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dfea00700 1 -- 172.21.15.3:0/2021722415 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e00165800 con 0x7f8e0006ccf0 2024-09-13T16:51:15.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dfea00700 1 --2- 172.21.15.3:0/2021722415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0006ccf0 0x7f8e001472c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009910 tx=0x7f8dec009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973cdd70dceb3c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:15.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dfe000700 1 -- 172.21.15.3:0/2021722415 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec00ba60 con 0x7f8e0006ccf0 2024-09-13T16:51:15.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dfe000700 1 -- 172.21.15.3:0/2021722415 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec00bbc0 con 0x7f8e0006ccf0 2024-09-13T16:51:15.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8dfe000700 1 -- 172.21.15.3:0/2021722415 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec009e60 con 0x7f8e0006ccf0 2024-09-13T16:51:15.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8e04e00700 1 -- 172.21.15.3:0/2021722415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0006ccf0 msgr2=0x7f8e001472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8e04e00700 1 --2- 172.21.15.3:0/2021722415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0006ccf0 0x7f8e001472c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009910 tx=0x7f8dec009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:15.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8e04e00700 1 -- 172.21.15.3:0/2021722415 shutdown_connections 2024-09-13T16:51:15.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8e04e00700 1 --2- 172.21.15.3:0/2021722415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8e04e00700 1 --2- 172.21.15.3:0/2021722415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0006ccf0 0x7f8e001472c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8e04e00700 1 -- 172.21.15.3:0/2021722415 >> 172.21.15.3:0/2021722415 conn(0x7f8e000b0cf0 msgr2=0x7f8e000b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:15.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8e04e00700 1 -- 172.21.15.3:0/2021722415 shutdown_connections 2024-09-13T16:51:15.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.405+0000 7f8e04e00700 1 -- 172.21.15.3:0/2021722415 wait complete. 2024-09-13T16:51:15.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.409+0000 7f8e04e00700 1 Processor -- start 2024-09-13T16:51:15.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.409+0000 7f8e04e00700 1 -- start start 2024-09-13T16:51:15.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.409+0000 7f8e04e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0006ccf0 0x7f8e0015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.409+0000 7f8e04e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.409+0000 7f8e04e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e00146ee0 0x7f8e0015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.409+0000 7f8e04e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000b2730 con 0x7f8e00146ee0 2024-09-13T16:51:15.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.409+0000 7f8e04e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0015b8b0 con 0x7f8e0006ccf0 2024-09-13T16:51:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.409+0000 7f8e04e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001610f0 con 0x7f8e0006fea0 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dffe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e00146ee0 0x7f8e0015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dffe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e00146ee0 0x7f8e0015bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41348/0 (socket says 172.21.15.3:41348) 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dffe00700 1 -- 172.21.15.3:0/946859576 learned_addr learned my addr 172.21.15.3:0/946859576 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dfea00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dff400700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0006ccf0 0x7f8e0015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dfea00700 1 -- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0006ccf0 msgr2=0x7f8e0015b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dfea00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0006ccf0 0x7f8e0015b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dfea00700 1 -- 172.21.15.3:0/946859576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e00146ee0 msgr2=0x7f8e0015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dfea00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e00146ee0 0x7f8e0015bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dfea00700 1 -- 172.21.15.3:0/946859576 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec009570 con 0x7f8e0006fea0 2024-09-13T16:51:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dffe00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e00146ee0 0x7f8e0015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8dfea00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00160bb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8dec016000 tx=0x7f8dec004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8df7400700 1 -- 172.21.15.3:0/946859576 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec004450 con 0x7f8e0006fea0 2024-09-13T16:51:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8df7400700 1 -- 172.21.15.3:0/946859576 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec01f070 con 0x7f8e0006fea0 2024-09-13T16:51:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e000b2170 con 0x7f8e0006fea0 2024-09-13T16:51:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8df7400700 1 -- 172.21.15.3:0/946859576 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec003900 con 0x7f8e0006fea0 2024-09-13T16:51:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.413+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e00144ba0 con 0x7f8e0006fea0 2024-09-13T16:51:15.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.417+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e001472c0 con 0x7f8e0006fea0 2024-09-13T16:51:15.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.417+0000 7f8df7400700 1 -- 172.21.15.3:0/946859576 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8dec003a60 con 0x7f8e0006fea0 2024-09-13T16:51:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.417+0000 7f8df7400700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8de0066730 0x7f8de0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.417+0000 7f8df7400700 1 -- 172.21.15.3:0/946859576 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8dec021070 con 0x7f8e0006fea0 2024-09-13T16:51:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.421+0000 7f8dff400700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8de0066730 0x7f8de0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.421+0000 7f8dff400700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8de0066730 0x7f8de0068bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8df0000f60 tx=0x7f8df0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.433+0000 7f8df7400700 1 -- 172.21.15.3:0/946859576 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8dec052c40 con 0x7f8e0006fea0 2024-09-13T16:51:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.673+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8e000ae620 con 0x7f8e0006fea0 2024-09-13T16:51:15.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.673+0000 7f8df7400700 1 -- 172.21.15.3:0/946859576 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8dec015070 con 0x7f8e0006fea0 2024-09-13T16:51:15.679 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8de0066730 msgr2=0x7f8de0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8de0066730 0x7f8de0068bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8df0000f60 tx=0x7f8df0009040 comp rx=0 tx=0).stop 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 msgr2=0x7f8e00160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00160bb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8dec016000 tx=0x7f8dec004770 comp rx=0 tx=0).stop 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 shutdown_connections 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8de0066730 0x7f8de0068bf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e00146ee0 0x7f8e0015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006fea0 0x7f8e00160bb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 --2- 172.21.15.3:0/946859576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0006ccf0 0x7f8e0015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 >> 172.21.15.3:0/946859576 conn(0x7f8e000b0cf0 msgr2=0x7f8e00161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 shutdown_connections 2024-09-13T16:51:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:15.677+0000 7f8e04e00700 1 -- 172.21.15.3:0/946859576 wait complete. 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:51:15.704 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-13T16:51:15.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:15.704 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-13T16:51:15.721 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:51:15.721 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:51:15.721 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:51:15.722 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42389sec preferred_lft 42389sec 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:4d:e8:d2:67 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether f2:5f:fd:43:95:5c brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::4cad:7aff:fedc:96c0/64 scope link 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether f2:5f:fd:43:95:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f05f:fdff:fe43:955c/64 scope link 2024-09-13T16:51:15.724 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:15.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:15.725 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:51:15.802 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-13T16:51:15.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:15.805 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-13T16:51:15.868 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-13T16:51:15.868 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:51:15.868 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T16:51:15.869 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:51:15.869 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T16:51:15.869 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:51:15.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:15.869 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:15.869 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:15.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T16:51:15.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T16:51:15.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T16:51:15.870 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:16.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.006 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:51:16.022 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T16:51:16.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.024 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-13T16:51:16.085 INFO:teuthology.orchestra.run.smithi003.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-13T16:51:16.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.087 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:16.087 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:16.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T16:51:16.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-13T16:51:16.087 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:16.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 9a:24:c3:0d:99:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T16:51:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::9824:c3ff:fe0d:990f/64 scope link 2024-09-13T16:51:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:16.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T16:51:16.317 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:51:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:51:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:51:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-13T16:51:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:16.317 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T16:51:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 9a:24:c3:0d:99:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T16:51:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T16:51:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::9824:c3ff:fe0d:990f/64 scope link 2024-09-13T16:51:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:16.321 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-13T16:51:16.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.321 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:16.321 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:16.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-13T16:51:16.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T16:51:16.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-13T16:51:16.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-13T16:51:16.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-13T16:51:16.322 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:16.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.504 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:51:16.504 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:51:16.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.1 up 2024-09-13T16:51:16.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T16:51:16.504 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:51:16.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:51:16.583 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-13T16:51:16.598 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-13T16:51:16.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:51:16.598 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-13T16:51:16.655 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-13T16:51:16.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:16.787 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:16.787 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:51:16.815 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:16.815 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:51:16.815 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-13T16:51:16.815 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 2@61456116-a42d-4065-9bf5-a1daf0be594e.cephfs=/ 2024-09-13T16:51:16.815 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.3:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-13T16:51:16.823 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T16:51:16.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-13T16:51:16.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.857 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:16.922 INFO:teuthology.orchestra.run.smithi003.stdout:client4673 2024-09-13T16:51:16.923 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:51:16.923 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:16.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:16.986 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:17.053 INFO:teuthology.orchestra.run.smithi003.stdout:client4673 2024-09-13T16:51:17.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:17.054 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:17.127 INFO:teuthology.orchestra.run.smithi003.stdout:client4673 2024-09-13T16:51:17.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:17.127 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/61456116-a42d-4065-9bf5-a1daf0be594e.client4673/status') 2024-09-13T16:51:17.214 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4673 (3)172.21.15.3:0/1030781994 2024-09-13T16:51:17.228 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-09-13T16:51:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-09-13T16:51:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-09-13T16:51:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 0.000178631 s, 386 kB/s 2024-09-13T16:51:17.229 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-13T16:51:17.229 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:51:17.229 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T16:51:17.229 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:51:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.440+0000 7f6d90c00700 1 Processor -- start 2024-09-13T16:51:17.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.444+0000 7f6d90c00700 1 -- start start 2024-09-13T16:51:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.444+0000 7f6d90c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d8c1590d0 0x7f6d8c06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.444+0000 7f6d90c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c0cca20 0x7f6d8c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.444+0000 7f6d90c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6d8c06a390 0x7f6d8c15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.444+0000 7f6d90c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c0b2060 con 0x7f6d8c1590d0 2024-09-13T16:51:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.444+0000 7f6d90c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c0b21f0 con 0x7f6d8c06a390 2024-09-13T16:51:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.444+0000 7f6d90c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c06a930 con 0x7f6d8c0cca20 2024-09-13T16:51:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d8c1590d0 0x7f6d8c06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d8c1590d0 0x7f6d8c06ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41358/0 (socket says 172.21.15.3:41358) 2024-09-13T16:51:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 -- 172.21.15.3:0/4170184373 learned_addr learned my addr 172.21.15.3:0/4170184373 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8be00700 1 --2- 172.21.15.3:0/4170184373 >> v2:172.21.15.157:3300/0 conn(0x7f6d8c06a390 0x7f6d8c15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d82a00700 1 --2- 172.21.15.3:0/4170184373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c0cca20 0x7f6d8c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:17.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 -- 172.21.15.3:0/4170184373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c0cca20 msgr2=0x7f6d8c06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:17.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 --2- 172.21.15.3:0/4170184373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c0cca20 0x7f6d8c06d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 -- 172.21.15.3:0/4170184373 >> v2:172.21.15.157:3300/0 conn(0x7f6d8c06a390 msgr2=0x7f6d8c15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:17.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 --2- 172.21.15.3:0/4170184373 >> v2:172.21.15.157:3300/0 conn(0x7f6d8c06a390 0x7f6d8c15ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 -- 172.21.15.3:0/4170184373 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c06ac10 con 0x7f6d8c1590d0 2024-09-13T16:51:17.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8b400700 1 --2- 172.21.15.3:0/4170184373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c06ce70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c06a840 tx=0x7f6d7c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01edab632d91c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:17.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8aa00700 1 -- 172.21.15.3:0/4170184373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c011830 con 0x7f6d8c1590d0 2024-09-13T16:51:17.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8aa00700 1 -- 172.21.15.3:0/4170184373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c011990 con 0x7f6d8c1590d0 2024-09-13T16:51:17.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d8aa00700 1 -- 172.21.15.3:0/4170184373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c011d20 con 0x7f6d8c1590d0 2024-09-13T16:51:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 -- 172.21.15.3:0/4170184373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 msgr2=0x7f6d8c06ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 --2- 172.21.15.3:0/4170184373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c06ce70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c06a840 tx=0x7f6d7c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 -- 172.21.15.3:0/4170184373 shutdown_connections 2024-09-13T16:51:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 --2- 172.21.15.3:0/4170184373 >> v2:172.21.15.157:3300/0 conn(0x7f6d8c06a390 0x7f6d8c15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 --2- 172.21.15.3:0/4170184373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c0cca20 0x7f6d8c06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 --2- 172.21.15.3:0/4170184373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c06ce70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 -- 172.21.15.3:0/4170184373 >> 172.21.15.3:0/4170184373 conn(0x7f6d8c0b0cf0 msgr2=0x7f6d8c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:17.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 -- 172.21.15.3:0/4170184373 shutdown_connections 2024-09-13T16:51:17.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.448+0000 7f6d90c00700 1 -- 172.21.15.3:0/4170184373 wait complete. 2024-09-13T16:51:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.452+0000 7f6d90c00700 1 Processor -- start 2024-09-13T16:51:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.452+0000 7f6d90c00700 1 -- start start 2024-09-13T16:51:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.452+0000 7f6d90c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c06a390 0x7f6d8c0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d90c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c0cca20 0x7f6d8c1f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d90c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d90c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c15b6b0 con 0x7f6d8c1590d0 2024-09-13T16:51:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d90c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1f6910 con 0x7f6d8c0cca20 2024-09-13T16:51:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d90c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1f6bf0 con 0x7f6d8c06a390 2024-09-13T16:51:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41370/0 (socket says 172.21.15.3:41370) 2024-09-13T16:51:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 -- 172.21.15.3:0/1093932893 learned_addr learned my addr 172.21.15.3:0/1093932893 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d82a00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c0cca20 0x7f6d8c1f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8b400700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c06a390 0x7f6d8c0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 -- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c06a390 msgr2=0x7f6d8c0b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c06a390 0x7f6d8c0b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 -- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c0cca20 msgr2=0x7f6d8c1f5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c0cca20 0x7f6d8c1f5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 -- 172.21.15.3:0/1093932893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c009570 con 0x7f6d8c1590d0 2024-09-13T16:51:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8b400700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c06a390 0x7f6d8c0b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d8be00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c159630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400eed0 tx=0x7f6d8400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d83e00700 1 -- 172.21.15.3:0/1093932893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84010040 con 0x7f6d8c1590d0 2024-09-13T16:51:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c15b970 con 0x7f6d8c1590d0 2024-09-13T16:51:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d83e00700 1 -- 172.21.15.3:0/1093932893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8400baa0 con 0x7f6d8c1590d0 2024-09-13T16:51:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d83e00700 1 -- 172.21.15.3:0/1093932893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400be50 con 0x7f6d8c1590d0 2024-09-13T16:51:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.456+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c0b3210 con 0x7f6d8c1590d0 2024-09-13T16:51:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.460+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c06b580 con 0x7f6d8c1590d0 2024-09-13T16:51:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.460+0000 7f6d83e00700 1 -- 172.21.15.3:0/1093932893 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d84014400 con 0x7f6d8c1590d0 2024-09-13T16:51:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.460+0000 7f6d83e00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d68066730 0x7f6d68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.460+0000 7f6d83e00700 1 -- 172.21.15.3:0/1093932893 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6d84084790 con 0x7f6d8c1590d0 2024-09-13T16:51:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.460+0000 7f6d8b400700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d68066730 0x7f6d68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:17.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.464+0000 7f6d8b400700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d68066730 0x7f6d68068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009c30 tx=0x7f6d7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.468+0000 7f6d83e00700 1 -- 172.21.15.3:0/1093932893 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6d840516b0 con 0x7f6d8c1590d0 2024-09-13T16:51:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.700+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6d8c0b3210 con 0x7f6d8c1590d0 2024-09-13T16:51:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.700+0000 7f6d83e00700 1 -- 172.21.15.3:0/1093932893 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6d8c0b3210 con 0x7f6d8c1590d0 2024-09-13T16:51:17.706 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T16:51:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d68066730 msgr2=0x7f6d68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d68066730 0x7f6d68068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009c30 tx=0x7f6d7c008030 comp rx=0 tx=0).stop 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 msgr2=0x7f6d8c159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c159630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400eed0 tx=0x7f6d8400ef00 comp rx=0 tx=0).stop 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 shutdown_connections 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d68066730 0x7f6d68068bf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d8c1590d0 0x7f6d8c159630 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c0cca20 0x7f6d8c1f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 --2- 172.21.15.3:0/1093932893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c06a390 0x7f6d8c0b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 >> 172.21.15.3:0/1093932893 conn(0x7f6d8c0b0cf0 msgr2=0x7f6d8c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 shutdown_connections 2024-09-13T16:51:17.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:17.704+0000 7f6d90c00700 1 -- 172.21.15.3:0/1093932893 wait complete. 2024-09-13T16:51:17.734 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T16:51:17.735 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:51:17.735 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-13T16:51:17.735 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T16:51:17.735 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T16:51:17.735 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T16:51:17.735 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:51:17.735 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:51:17.735 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T16:51:17.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:17.735 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:17.755 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42409sec preferred_lft 42409sec 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:51:17.756 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:84:1d:d7:cc brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:17.757 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:51:17.757 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:17.757 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T16:51:17.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:17.758 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:17.758 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:17.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-09-13T16:51:17.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-09-13T16:51:17.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-09-13T16:51:17.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T16:51:17.758 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:17.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:17.908 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:51:17.939 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-13T16:51:17.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:17.941 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-13T16:51:18.023 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-13T16:51:18.023 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:51:18.023 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T16:51:18.023 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T16:51:18.023 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:51:18.023 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-09-13T16:51:18.023 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T16:51:18.023 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:51:18.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:18.024 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:18.024 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:18.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T16:51:18.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T16:51:18.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T16:51:18.024 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:18.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:18.146 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-13T16:51:18.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:18.162 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-09-13T16:51:18.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:18.223 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:18.223 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:18.223 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T16:51:18.223 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-13T16:51:18.223 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:18.322 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-13T16:51:18.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:18.323 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:18.323 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:18.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-13T16:51:18.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T16:51:18.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-13T16:51:18.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-13T16:51:18.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-13T16:51:18.323 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:18.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:18.495 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:18.495 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:18.495 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-13T16:51:18.495 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T16:51:18.495 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:18.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:51:18.556 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T16:51:18.571 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T16:51:18.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:51:18.572 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T16:51:18.626 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-13T16:51:18.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:18.627 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:18.756 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:18.756 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:51:19.492 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:19.492 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:51:19.492 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-13T16:51:19.492 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@61456116-a42d-4065-9bf5-a1daf0be594e.cephfs=/ 2024-09-13T16:51:19.492 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.3:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-13T16:51:19.499 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T16:51:19.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:19.499 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T16:51:19.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:19.535 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:19.610 INFO:teuthology.orchestra.run.smithi157.stdout:client4909 2024-09-13T16:51:19.611 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:51:19.611 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:19.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:19.670 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:19.734 INFO:teuthology.orchestra.run.smithi157.stdout:client4909 2024-09-13T16:51:19.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:19.735 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:19.800 INFO:teuthology.orchestra.run.smithi157.stdout:client4909 2024-09-13T16:51:19.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:19.801 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/61456116-a42d-4065-9bf5-a1daf0be594e.client4909/status') 2024-09-13T16:51:19.884 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4909 (3)192.168.0.1:0/907471068 2024-09-13T16:51:19.884 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2024-09-13T16:51:19.885 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2024-09-13T16:51:19.885 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2024-09-13T16:51:19.885 INFO:teuthology.orchestra.run.smithi157.stderr:68 bytes copied, 0.000168847 s, 403 kB/s 2024-09-13T16:51:19.888 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-13T16:51:19.888 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:51:19.888 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-13T16:51:19.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:51:20.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d6800700 1 Processor -- start 2024-09-13T16:51:20.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d6800700 1 -- start start 2024-09-13T16:51:20.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d6800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0144790 0x7f66d006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:20.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d6800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66d006cb20 0x7f66d015b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:20.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d6800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d00b4100 con 0x7f66d01443b0 2024-09-13T16:51:20.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d6800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d006cf90 con 0x7f66d006cb20 2024-09-13T16:51:20.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d6800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d00ce8e0 con 0x7f66d0144790 2024-09-13T16:51:20.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d5400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66d01443b0 0x7f66d00b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56820/0 (socket says 172.21.15.3:56820) 2024-09-13T16:51:20.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d5400700 1 -- 172.21.15.3:0/4201569057 learned_addr learned my addr 172.21.15.3:0/4201569057 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:20.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d4a00700 1 --2- 172.21.15.3:0/4201569057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0144790 0x7f66d006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.060+0000 7f66d5e00700 1 --2- 172.21.15.3:0/4201569057 >> v2:172.21.15.157:3300/0 conn(0x7f66d006cb20 0x7f66d015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66cfe00700 1 -- 172.21.15.3:0/4201569057 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121278899 0 0) 0x7f66d00b4100 con 0x7f66d01443b0 2024-09-13T16:51:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66cfe00700 1 -- 172.21.15.3:0/4201569057 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b80035d0 con 0x7f66d01443b0 2024-09-13T16:51:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d4a00700 1 -- 172.21.15.3:0/4201569057 >> v2:172.21.15.157:3300/0 conn(0x7f66d006cb20 msgr2=0x7f66d015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d4a00700 1 --2- 172.21.15.3:0/4201569057 >> v2:172.21.15.157:3300/0 conn(0x7f66d006cb20 0x7f66d015b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d4a00700 1 -- 172.21.15.3:0/4201569057 >> v1:172.21.15.3:6789/0 conn(0x7f66d01443b0 legacy=0x7f66d00b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d4a00700 1 -- 172.21.15.3:0/4201569057 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d00ccd30 con 0x7f66d0144790 2024-09-13T16:51:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d4a00700 1 --2- 172.21.15.3:0/4201569057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0144790 0x7f66d006c5e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009850 tx=0x7f66bc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5be5e05332769d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66cfe00700 1 -- 172.21.15.3:0/4201569057 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4001670 con 0x7f66d0144790 2024-09-13T16:51:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66cfe00700 1 -- 172.21.15.3:0/4201569057 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc010590 con 0x7f66d0144790 2024-09-13T16:51:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66cfe00700 1 -- 172.21.15.3:0/4201569057 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc010940 con 0x7f66d0144790 2024-09-13T16:51:20.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d6800700 1 -- 172.21.15.3:0/4201569057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0144790 msgr2=0x7f66d006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:20.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d6800700 1 --2- 172.21.15.3:0/4201569057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0144790 0x7f66d006c5e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009850 tx=0x7f66bc009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d6800700 1 -- 172.21.15.3:0/4201569057 shutdown_connections 2024-09-13T16:51:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d6800700 1 --2- 172.21.15.3:0/4201569057 >> v2:172.21.15.157:3300/0 conn(0x7f66d006cb20 0x7f66d015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d6800700 1 --2- 172.21.15.3:0/4201569057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0144790 0x7f66d006c5e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d6800700 1 -- 172.21.15.3:0/4201569057 >> 172.21.15.3:0/4201569057 conn(0x7f66d00b0cf0 msgr2=0x7f66d00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d6800700 1 -- 172.21.15.3:0/4201569057 shutdown_connections 2024-09-13T16:51:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.064+0000 7f66d6800700 1 -- 172.21.15.3:0/4201569057 wait complete. 2024-09-13T16:51:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.068+0000 7f66d6800700 1 Processor -- start 2024-09-13T16:51:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.068+0000 7f66d6800700 1 -- start start 2024-09-13T16:51:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d6800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d006cb20 0x7f66d0157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d01443b0 0x7f66d013d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d6800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0140160 0x7f66d013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d6800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d013d860 con 0x7f66d006cb20 2024-09-13T16:51:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d6800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0158260 con 0x7f66d01443b0 2024-09-13T16:51:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d6800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d01563a0 con 0x7f66d0140160 2024-09-13T16:51:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d006cb20 0x7f66d0157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d006cb20 0x7f66d0157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41384/0 (socket says 172.21.15.3:41384) 2024-09-13T16:51:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 -- 172.21.15.3:0/1935175060 learned_addr learned my addr 172.21.15.3:0/1935175060 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d4a00700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d01443b0 0x7f66d013d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5e00700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0140160 0x7f66d013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 -- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0140160 msgr2=0x7f66d013afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0140160 0x7f66d013afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 -- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d01443b0 msgr2=0x7f66d013d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d01443b0 0x7f66d013d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 -- 172.21.15.3:0/1935175060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0094b0 con 0x7f66d006cb20 2024-09-13T16:51:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d4a00700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d01443b0 0x7f66d013d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:20.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d5400700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d006cb20 0x7f66d0157b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f66c400cac0 tx=0x7f66c400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:20.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66cd600700 1 -- 172.21.15.3:0/1935175060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4004a90 con 0x7f66d006cb20 2024-09-13T16:51:20.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66cd600700 1 -- 172.21.15.3:0/1935175060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c4004bf0 con 0x7f66d006cb20 2024-09-13T16:51:20.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d0156680 con 0x7f66d006cb20 2024-09-13T16:51:20.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66cd600700 1 -- 172.21.15.3:0/1935175060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c400ce40 con 0x7f66d006cb20 2024-09-13T16:51:20.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.072+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d0156c40 con 0x7f66d006cb20 2024-09-13T16:51:20.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.076+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d00cca20 con 0x7f66d006cb20 2024-09-13T16:51:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.076+0000 7f66cd600700 1 -- 172.21.15.3:0/1935175060 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66c40070d0 con 0x7f66d006cb20 2024-09-13T16:51:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.080+0000 7f66cd600700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f66b0066730 0x7f66b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.080+0000 7f66cd600700 1 -- 172.21.15.3:0/1935175060 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f66c40837a0 con 0x7f66d006cb20 2024-09-13T16:51:20.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.080+0000 7f66d4a00700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f66b0066730 0x7f66b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.080+0000 7f66d4a00700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f66b0066730 0x7f66b0068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66d006fc80 tx=0x7f66bc008000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:20.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.088+0000 7f66cd600700 1 -- 172.21.15.3:0/1935175060 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f66c4050630 con 0x7f66d006cb20 2024-09-13T16:51:20.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.300+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f66d0143960 con 0x7f66d006cb20 2024-09-13T16:51:20.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.300+0000 7f66cd600700 1 -- 172.21.15.3:0/1935175060 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f66c4053c50 con 0x7f66d006cb20 2024-09-13T16:51:20.306 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T16:51:20.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f66b0066730 msgr2=0x7f66b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:20.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f66b0066730 0x7f66b0068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66d006fc80 tx=0x7f66bc008000 comp rx=0 tx=0).stop 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d006cb20 msgr2=0x7f66d0157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d006cb20 0x7f66d0157b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f66c400cac0 tx=0x7f66c400cdd0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 shutdown_connections 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f66b0066730 0x7f66b0068bf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66d0140160 0x7f66d013afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d01443b0 0x7f66d013d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 --2- 172.21.15.3:0/1935175060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d006cb20 0x7f66d0157b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 >> 172.21.15.3:0/1935175060 conn(0x7f66d00b0cf0 msgr2=0x7f66d00afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 shutdown_connections 2024-09-13T16:51:20.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:20.304+0000 7f66d6800700 1 -- 172.21.15.3:0/1935175060 wait complete. 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:51:20.332 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-13T16:51:20.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.333 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-13T16:51:20.353 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:51:20.353 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:51:20.353 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:51:20.353 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.353 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-13T16:51:20.353 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42406sec preferred_lft 42406sec 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-13T16:51:20.354 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:84:1d:d7:cc brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether fa:68:c5:28:5b:57 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::7c57:89ff:fe7a:1617/64 scope link 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether fa:68:c5:28:5b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T16:51:20.355 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::f868:c5ff:fe28:5b57/64 scope link 2024-09-13T16:51:20.356 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.357 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:51:20.434 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-13T16:51:20.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.437 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-13T16:51:20.500 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-13T16:51:20.500 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:51:20.500 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T16:51:20.500 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T16:51:20.500 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:51:20.501 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-09-13T16:51:20.501 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T16:51:20.501 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:51:20.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.502 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:20.502 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:20.502 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T16:51:20.502 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T16:51:20.502 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T16:51:20.502 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:20.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.631 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-13T16:51:20.648 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T16:51:20.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.650 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-09-13T16:51:20.709 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-13T16:51:20.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.710 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:20.710 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:20.710 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-13T16:51:20.710 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-13T16:51:20.710 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:20.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.825 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-13T16:51:20.862 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:51:20.862 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:51:20.862 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:51:20.862 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.862 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-13T16:51:20.862 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.863 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T16:51:20.863 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 7e:14:8a:73:fc:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T16:51:20.863 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T16:51:20.863 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.863 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::7c14:8aff:fe73:fcf3/64 scope link 2024-09-13T16:51:20.863 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.865 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-13T16:51:20.948 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:51:20.948 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:51:20.948 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 7e:14:8a:73:fc:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::7c14:8aff:fe73:fcf3/64 scope link 2024-09-13T16:51:20.949 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:51:20.953 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-13T16:51:20.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:20.953 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:20.953 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:20.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-13T16:51:20.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T16:51:20.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-13T16:51:20.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-13T16:51:20.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-13T16:51:20.954 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:21.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:21.127 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-09-13T16:51:21.128 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-13T16:51:21.128 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2024-09-13T16:51:21.128 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T16:51:21.128 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-13T16:51:21.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:51:21.194 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-13T16:51:21.209 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-13T16:51:21.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:51:21.210 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-13T16:51:21.271 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-13T16:51:21.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:21.271 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:21.396 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:21.396 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:51:21.425 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:51:21.425 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:51:21.425 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-13T16:51:21.425 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 3@61456116-a42d-4065-9bf5-a1daf0be594e.cephfs=/ 2024-09-13T16:51:21.425 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.3:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-13T16:51:21.432 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T16:51:21.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:21.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-13T16:51:21.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:21.459 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:21.526 INFO:teuthology.orchestra.run.smithi157.stdout:client4911 2024-09-13T16:51:21.527 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:51:21.527 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:21.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:21.588 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:21.658 INFO:teuthology.orchestra.run.smithi157.stdout:client4911 2024-09-13T16:51:21.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:21.659 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:51:21.728 INFO:teuthology.orchestra.run.smithi157.stdout:client4911 2024-09-13T16:51:21.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:21.728 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/61456116-a42d-4065-9bf5-a1daf0be594e.client4911/status') 2024-09-13T16:51:21.807 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4911 (3)172.21.15.157:0/1723980088 2024-09-13T16:51:21.807 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2024-09-13T16:51:21.808 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2024-09-13T16:51:21.808 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2024-09-13T16:51:21.808 INFO:teuthology.orchestra.run.smithi157.stderr:71 bytes copied, 0.00017607 s, 403 kB/s 2024-09-13T16:51:21.811 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-13T16:51:21.829 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:22.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.004+0000 7f6187e00700 1 Processor -- start 2024-09-13T16:51:22.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.004+0000 7f6187e00700 1 -- start start 2024-09-13T16:51:22.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6187e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815d1b0 0x7f61880d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6187e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61880d1510 con 0x7f61880d25b0 2024-09-13T16:51:22.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6187e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61880d17f0 con 0x7f618815cdd0 2024-09-13T16:51:22.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6187e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880d1ad0 con 0x7f618815d1b0 2024-09-13T16:51:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6187400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f61880d25b0 0x7f61880cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56822/0 (socket says 172.21.15.3:56822) 2024-09-13T16:51:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6187400700 1 -- 172.21.15.3:0/2819175339 learned_addr learned my addr 172.21.15.3:0/2819175339 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6186000700 1 --2- 172.21.15.3:0/2819175339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815d1b0 0x7f61880d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6185600700 1 -- 172.21.15.3:0/2819175339 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230882078 0 0) 0x7f61880d1510 con 0x7f61880d25b0 2024-09-13T16:51:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6185600700 1 -- 172.21.15.3:0/2819175339 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f616c0035d0 con 0x7f61880d25b0 2024-09-13T16:51:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6185600700 1 -- 172.21.15.3:0/2819175339 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784766153 0 0) 0x7f61880d17f0 con 0x7f618815cdd0 2024-09-13T16:51:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6185600700 1 -- 172.21.15.3:0/2819175339 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61880d1510 con 0x7f618815cdd0 2024-09-13T16:51:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6186000700 1 -- 172.21.15.3:0/2819175339 >> v1:172.21.15.157:6789/0 conn(0x7f618815cdd0 legacy=0x7f61880b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6186000700 1 -- 172.21.15.3:0/2819175339 >> v1:172.21.15.3:6789/0 conn(0x7f61880d25b0 legacy=0x7f61880cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6186000700 1 -- 172.21.15.3:0/2819175339 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880d1db0 con 0x7f618815d1b0 2024-09-13T16:51:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.008+0000 7f6186000700 1 --2- 172.21.15.3:0/2819175339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815d1b0 0x7f61880d0e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6174009880 tx=0x7f6174009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d899a59f3c30bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6185600700 1 -- 172.21.15.3:0/2819175339 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174011830 con 0x7f618815d1b0 2024-09-13T16:51:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6185600700 1 -- 172.21.15.3:0/2819175339 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174011990 con 0x7f618815d1b0 2024-09-13T16:51:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6185600700 1 -- 172.21.15.3:0/2819175339 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174011d20 con 0x7f618815d1b0 2024-09-13T16:51:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 -- 172.21.15.3:0/2819175339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815d1b0 msgr2=0x7f61880d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 --2- 172.21.15.3:0/2819175339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815d1b0 0x7f61880d0e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6174009880 tx=0x7f6174009b90 comp rx=0 tx=0).stop 2024-09-13T16:51:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 -- 172.21.15.3:0/2819175339 shutdown_connections 2024-09-13T16:51:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 --2- 172.21.15.3:0/2819175339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815d1b0 0x7f61880d0e10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 -- 172.21.15.3:0/2819175339 >> 172.21.15.3:0/2819175339 conn(0x7f61880d3190 msgr2=0x7f61880d35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 -- 172.21.15.3:0/2819175339 shutdown_connections 2024-09-13T16:51:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 -- 172.21.15.3:0/2819175339 wait complete. 2024-09-13T16:51:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 Processor -- start 2024-09-13T16:51:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.012+0000 7f6187e00700 1 -- start start 2024-09-13T16:51:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d25b0 0x7f6188150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815cdd0 0x7f61880cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f618815d1b0 0x7f61880cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880ca810 con 0x7f61880d25b0 2024-09-13T16:51:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880c8e60 con 0x7f618815d1b0 2024-09-13T16:51:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880c9ee0 con 0x7f618815cdd0 2024-09-13T16:51:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d25b0 0x7f6188150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d25b0 0x7f6188150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41396/0 (socket says 172.21.15.3:41396) 2024-09-13T16:51:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 -- 172.21.15.3:0/1162312508 learned_addr learned my addr 172.21.15.3:0/1162312508 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187400700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f618815d1b0 0x7f61880cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186000700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815cdd0 0x7f61880cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 -- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815cdd0 msgr2=0x7f61880cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815cdd0 0x7f61880cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 -- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f618815d1b0 msgr2=0x7f61880cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f618815d1b0 0x7f61880cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 -- 172.21.15.3:0/1162312508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61740094b0 con 0x7f61880d25b0 2024-09-13T16:51:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6186a00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d25b0 0x7f6188150300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6178001f00 tx=0x7f617800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f617ea00700 1 -- 172.21.15.3:0/1162312508 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178002100 con 0x7f61880d25b0 2024-09-13T16:51:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f617ea00700 1 -- 172.21.15.3:0/1162312508 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617800ade0 con 0x7f61880d25b0 2024-09-13T16:51:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f617ea00700 1 -- 172.21.15.3:0/1162312508 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178003c60 con 0x7f61880d25b0 2024-09-13T16:51:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61880d2be0 con 0x7f61880d25b0 2024-09-13T16:51:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.016+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61880ca4a0 con 0x7f61880d25b0 2024-09-13T16:51:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.020+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61880d0e10 con 0x7f61880d25b0 2024-09-13T16:51:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.020+0000 7f617ea00700 1 -- 172.21.15.3:0/1162312508 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61780050d0 con 0x7f61880d25b0 2024-09-13T16:51:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.024+0000 7f617ea00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6168066730 0x7f6168068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.024+0000 7f617ea00700 1 -- 172.21.15.3:0/1162312508 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6178084810 con 0x7f61880d25b0 2024-09-13T16:51:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.024+0000 7f6186000700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6168066730 0x7f6168068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.024+0000 7f6186000700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6168066730 0x7f6168068bf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f618815d5b0 tx=0x7f617401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.032+0000 7f617ea00700 1 -- 172.21.15.3:0/1162312508 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6178051720 con 0x7f61880d25b0 2024-09-13T16:51:22.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.220+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6188067730 con 0x7f61880d25b0 2024-09-13T16:51:22.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.224+0000 7f617ea00700 1 -- 172.21.15.3:0/1162312508 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f6178054d40 con 0x7f61880d25b0 2024-09-13T16:51:22.230 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:22.230 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6168066730 msgr2=0x7f6168068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6168066730 0x7f6168068bf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f618815d5b0 tx=0x7f617401c040 comp rx=0 tx=0).stop 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d25b0 msgr2=0x7f6188150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d25b0 0x7f6188150300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6178001f00 tx=0x7f617800a340 comp rx=0 tx=0).stop 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 shutdown_connections 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6168066730 0x7f6168068bf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f618815d1b0 0x7f61880cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f618815cdd0 0x7f61880cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 --2- 172.21.15.3:0/1162312508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d25b0 0x7f6188150300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 >> 172.21.15.3:0/1162312508 conn(0x7f61880d3190 msgr2=0x7f61880b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 shutdown_connections 2024-09-13T16:51:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.228+0000 7f6187e00700 1 -- 172.21.15.3:0/1162312508 wait complete. 2024-09-13T16:51:22.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:22.255 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:22.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.420+0000 7fd861800700 1 Processor -- start 2024-09-13T16:51:22.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.420+0000 7fd861800700 1 -- start start 2024-09-13T16:51:22.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd861800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd85c15e510 0x7fd85c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd861800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c151e80 con 0x7fd85c06bfd0 2024-09-13T16:51:22.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd861800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c15be90 con 0x7fd85c0b4f90 2024-09-13T16:51:22.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd861800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c0ceb80 con 0x7fd85c15e510 2024-09-13T16:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85be00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd85c06bfd0 0x7fd85c06ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56838/0 (socket says 172.21.15.3:56838) 2024-09-13T16:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85be00700 1 -- 172.21.15.3:0/437790344 learned_addr learned my addr 172.21.15.3:0/437790344 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85b400700 1 --2- 172.21.15.3:0/437790344 >> v2:172.21.15.157:3300/0 conn(0x7fd85c15e510 0x7fd85c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85aa00700 1 -- 172.21.15.3:0/437790344 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123417877 0 0) 0x7fd85c151e80 con 0x7fd85c06bfd0 2024-09-13T16:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85aa00700 1 -- 172.21.15.3:0/437790344 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8400035d0 con 0x7fd85c06bfd0 2024-09-13T16:51:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85aa00700 1 -- 172.21.15.3:0/437790344 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677896150 0 0) 0x7fd85c15be90 con 0x7fd85c0b4f90 2024-09-13T16:51:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85aa00700 1 -- 172.21.15.3:0/437790344 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd85c151e80 con 0x7fd85c0b4f90 2024-09-13T16:51:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85b400700 1 -- 172.21.15.3:0/437790344 >> v1:172.21.15.157:6789/0 conn(0x7fd85c0b4f90 legacy=0x7fd85c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85aa00700 1 -- 172.21.15.3:0/437790344 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1086450630 0 0) 0x7fd8400035d0 con 0x7fd85c06bfd0 2024-09-13T16:51:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85b400700 1 -- 172.21.15.3:0/437790344 >> v1:172.21.15.3:6789/0 conn(0x7fd85c06bfd0 legacy=0x7fd85c06ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.424+0000 7fd85b400700 1 -- 172.21.15.3:0/437790344 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c15ead0 con 0x7fd85c15e510 2024-09-13T16:51:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd85b400700 1 --2- 172.21.15.3:0/437790344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c15e510 0x7fd85c15b950 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd848009850 tx=0x7fd848009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe42f15740dd60bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd85aa00700 1 -- 172.21.15.3:0/437790344 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c001670 con 0x7fd85c15e510 2024-09-13T16:51:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd85aa00700 1 -- 172.21.15.3:0/437790344 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848004030 con 0x7fd85c15e510 2024-09-13T16:51:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd85aa00700 1 -- 172.21.15.3:0/437790344 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480043c0 con 0x7fd85c15e510 2024-09-13T16:51:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd861800700 1 -- 172.21.15.3:0/437790344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c15e510 msgr2=0x7fd85c15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd861800700 1 --2- 172.21.15.3:0/437790344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c15e510 0x7fd85c15b950 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd848009850 tx=0x7fd848009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd861800700 1 -- 172.21.15.3:0/437790344 shutdown_connections 2024-09-13T16:51:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd861800700 1 --2- 172.21.15.3:0/437790344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c15e510 0x7fd85c15b950 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd861800700 1 -- 172.21.15.3:0/437790344 >> 172.21.15.3:0/437790344 conn(0x7fd85c0cd5c0 msgr2=0x7fd85c070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd861800700 1 -- 172.21.15.3:0/437790344 shutdown_connections 2024-09-13T16:51:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.428+0000 7fd861800700 1 -- 172.21.15.3:0/437790344 wait complete. 2024-09-13T16:51:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd861800700 1 Processor -- start 2024-09-13T16:51:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd861800700 1 -- start start 2024-09-13T16:51:22.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd861800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c06bfd0 0x7fd85c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd861800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c0b4f90 0x7fd85c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd861800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c15e510 0x7fd85c0ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd861800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c1f5810 con 0x7fd85c15e510 2024-09-13T16:51:22.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd861800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c14f330 con 0x7fd85c0b4f90 2024-09-13T16:51:22.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd861800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c1f5a60 con 0x7fd85c06bfd0 2024-09-13T16:51:22.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd860e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c15e510 0x7fd85c0ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd860e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c15e510 0x7fd85c0ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41408/0 (socket says 172.21.15.3:41408) 2024-09-13T16:51:22.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd860e00700 1 -- 172.21.15.3:0/903626309 learned_addr learned my addr 172.21.15.3:0/903626309 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.432+0000 7fd85b400700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c0b4f90 0x7fd85c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd85be00700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c06bfd0 0x7fd85c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd860e00700 1 -- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c06bfd0 msgr2=0x7fd85c153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd860e00700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c06bfd0 0x7fd85c153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd860e00700 1 -- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c0b4f90 msgr2=0x7fd85c15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd860e00700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c0b4f90 0x7fd85c15d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd860e00700 1 -- 172.21.15.3:0/903626309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480094b0 con 0x7fd85c15e510 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd85be00700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c06bfd0 0x7fd85c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:22.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd85b400700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c0b4f90 0x7fd85c15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:22.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd860e00700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c15e510 0x7fd85c0ae9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd85400cbc0 tx=0x7fd85400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd853e00700 1 -- 172.21.15.3:0/903626309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854003c20 con 0x7fd85c15e510 2024-09-13T16:51:22.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd853e00700 1 -- 172.21.15.3:0/903626309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd854003da0 con 0x7fd85c15e510 2024-09-13T16:51:22.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c1f5d40 con 0x7fd85c15e510 2024-09-13T16:51:22.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd853e00700 1 -- 172.21.15.3:0/903626309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85400aae0 con 0x7fd85c15e510 2024-09-13T16:51:22.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c1f6300 con 0x7fd85c15e510 2024-09-13T16:51:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.436+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c15b950 con 0x7fd85c15e510 2024-09-13T16:51:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.440+0000 7fd853e00700 1 -- 172.21.15.3:0/903626309 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd85400ac40 con 0x7fd85c15e510 2024-09-13T16:51:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.440+0000 7fd853e00700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd83c066730 0x7fd83c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.440+0000 7fd853e00700 1 -- 172.21.15.3:0/903626309 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd854084650 con 0x7fd85c15e510 2024-09-13T16:51:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.440+0000 7fd85be00700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd83c066730 0x7fd83c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.444+0000 7fd85be00700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd83c066730 0x7fd83c068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd84c001eb0 tx=0x7fd84c001c80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.448+0000 7fd853e00700 1 -- 172.21.15.3:0/903626309 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8540514e0 con 0x7fd85c15e510 2024-09-13T16:51:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.636+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd85c1f64c0 con 0x7fd85c15e510 2024-09-13T16:51:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.636+0000 7fd853e00700 1 -- 172.21.15.3:0/903626309 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd854054b00 con 0x7fd85c15e510 2024-09-13T16:51:22.643 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd83c066730 msgr2=0x7fd83c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd83c066730 0x7fd83c068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd84c001eb0 tx=0x7fd84c001c80 comp rx=0 tx=0).stop 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c15e510 msgr2=0x7fd85c0ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c15e510 0x7fd85c0ae9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd85400cbc0 tx=0x7fd85400ced0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 shutdown_connections 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd83c066730 0x7fd83c068bf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c15e510 0x7fd85c0ae9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c0b4f90 0x7fd85c15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 --2- 172.21.15.3:0/903626309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c06bfd0 0x7fd85c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 >> 172.21.15.3:0/903626309 conn(0x7fd85c0cd5c0 msgr2=0x7fd85c0cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 shutdown_connections 2024-09-13T16:51:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.640+0000 7fd861800700 1 -- 172.21.15.3:0/903626309 wait complete. 2024-09-13T16:51:22.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:22.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.840+0000 7f48a5800700 1 Processor -- start 2024-09-13T16:51:22.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.840+0000 7f48a5800700 1 -- start start 2024-09-13T16:51:22.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f48a5800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a00aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f48a5800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48a00cf7a0 0x7f48a00ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f48a5800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a00af620 con 0x7f48a00b4e70 2024-09-13T16:51:22.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f48a5800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a006bd00 con 0x7f48a00cf7a0 2024-09-13T16:51:22.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f48a5800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a006af40 con 0x7f48a0159110 2024-09-13T16:51:22.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f4897e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f48a00b4e70 0x7f48a00cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56852/0 (socket says 172.21.15.3:56852) 2024-09-13T16:51:22.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f4897e00700 1 -- 172.21.15.3:0/1227768750 learned_addr learned my addr 172.21.15.3:0/1227768750 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:22.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f48a4e00700 1 --2- 172.21.15.3:0/1227768750 >> v2:172.21.15.3:3300/0 conn(0x7f48a00cf7a0 0x7f48a00ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f489fe00700 1 --2- 172.21.15.3:0/1227768750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a00aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f489f400700 1 -- 172.21.15.3:0/1227768750 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79815899 0 0) 0x7f4888001110 con 0x7f48a00b4e70 2024-09-13T16:51:22.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f489f400700 1 -- 172.21.15.3:0/1227768750 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48840035d0 con 0x7f48a00b4e70 2024-09-13T16:51:22.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f489fe00700 1 -- 172.21.15.3:0/1227768750 >> v2:172.21.15.3:3300/0 conn(0x7f48a00cf7a0 msgr2=0x7f48a00ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:22.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f489fe00700 1 --2- 172.21.15.3:0/1227768750 >> v2:172.21.15.3:3300/0 conn(0x7f48a00cf7a0 0x7f48a00ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f489fe00700 1 -- 172.21.15.3:0/1227768750 >> v1:172.21.15.3:6789/0 conn(0x7f48a00b4e70 legacy=0x7f48a00cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f489fe00700 1 -- 172.21.15.3:0/1227768750 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a006b220 con 0x7f48a0159110 2024-09-13T16:51:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.844+0000 7f48a4e00700 1 --2- 172.21.15.3:0/1227768750 >> v2:172.21.15.3:3300/0 conn(0x7f48a00cf7a0 0x7f48a00ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f489fe00700 1 --2- 172.21.15.3:0/1227768750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a00aed40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4890009850 tx=0x7f4890009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1714d9bb57723c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f489f400700 1 -- 172.21.15.3:0/1227768750 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890011830 con 0x7f48a0159110 2024-09-13T16:51:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f489f400700 1 -- 172.21.15.3:0/1227768750 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4890011990 con 0x7f48a0159110 2024-09-13T16:51:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f489f400700 1 -- 172.21.15.3:0/1227768750 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890011d40 con 0x7f48a0159110 2024-09-13T16:51:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f48a5800700 1 -- 172.21.15.3:0/1227768750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 msgr2=0x7f48a00aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f48a5800700 1 --2- 172.21.15.3:0/1227768750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a00aed40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4890009850 tx=0x7f4890009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f48a5800700 1 -- 172.21.15.3:0/1227768750 shutdown_connections 2024-09-13T16:51:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f48a5800700 1 --2- 172.21.15.3:0/1227768750 >> v2:172.21.15.3:3300/0 conn(0x7f48a00cf7a0 0x7f48a00ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f48a5800700 1 --2- 172.21.15.3:0/1227768750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a00aed40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f48a5800700 1 -- 172.21.15.3:0/1227768750 >> 172.21.15.3:0/1227768750 conn(0x7f48a00b2d50 msgr2=0x7f48a00b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f48a5800700 1 -- 172.21.15.3:0/1227768750 shutdown_connections 2024-09-13T16:51:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.848+0000 7f48a5800700 1 -- 172.21.15.3:0/1227768750 wait complete. 2024-09-13T16:51:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.852+0000 7f48a5800700 1 Processor -- start 2024-09-13T16:51:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.852+0000 7f48a5800700 1 -- start start 2024-09-13T16:51:22.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.852+0000 7f48a5800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a00b4e70 0x7f48a01524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.852+0000 7f48a5800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a00cf7a0 0x7f48a014c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.852+0000 7f48a5800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a014ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.852+0000 7f48a5800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a014a1f0 con 0x7f48a00cf7a0 2024-09-13T16:51:22.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.852+0000 7f48a5800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a014ad70 con 0x7f48a00b4e70 2024-09-13T16:51:22.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.852+0000 7f48a5800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a014b200 con 0x7f48a0159110 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a00cf7a0 0x7f48a014c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a00cf7a0 0x7f48a014c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41432/0 (socket says 172.21.15.3:41432) 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 -- 172.21.15.3:0/839905054 learned_addr learned my addr 172.21.15.3:0/839905054 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f489fe00700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a00b4e70 0x7f48a01524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f48a4e00700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a014ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 -- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 msgr2=0x7f48a014ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a014ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 -- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a00b4e70 msgr2=0x7f48a01524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a00b4e70 0x7f48a01524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 -- 172.21.15.3:0/839905054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489800ab80 con 0x7f48a00cf7a0 2024-09-13T16:51:22.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f4897e00700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a00cf7a0 0x7f48a014c770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f488800c760 tx=0x7f488800ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f489cc00700 1 -- 172.21.15.3:0/839905054 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888005d90 con 0x7f48a00cf7a0 2024-09-13T16:51:22.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f489cc00700 1 -- 172.21.15.3:0/839905054 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48880070d0 con 0x7f48a00cf7a0 2024-09-13T16:51:22.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f489cc00700 1 -- 172.21.15.3:0/839905054 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888007460 con 0x7f48a00cf7a0 2024-09-13T16:51:22.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48900094b0 con 0x7f48a00cf7a0 2024-09-13T16:51:22.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.856+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a014a720 con 0x7f48a00cf7a0 2024-09-13T16:51:22.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.860+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a014af00 con 0x7f48a00cf7a0 2024-09-13T16:51:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.860+0000 7f489cc00700 1 -- 172.21.15.3:0/839905054 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f48880075c0 con 0x7f48a00cf7a0 2024-09-13T16:51:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.860+0000 7f489cc00700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f487c066730 0x7f487c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.864+0000 7f489cc00700 1 -- 172.21.15.3:0/839905054 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f48880837c0 con 0x7f48a00cf7a0 2024-09-13T16:51:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.864+0000 7f489fe00700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f487c066730 0x7f487c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.864+0000 7f489fe00700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f487c066730 0x7f487c068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48900090b0 tx=0x7f489001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:22.868+0000 7f489cc00700 1 -- 172.21.15.3:0/839905054 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4888050780 con 0x7f48a00cf7a0 2024-09-13T16:51:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.052+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48a00af620 con 0x7f48a00cf7a0 2024-09-13T16:51:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.056+0000 7f489cc00700 1 -- 172.21.15.3:0/839905054 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48a00af620 con 0x7f48a00cf7a0 2024-09-13T16:51:23.060 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f487c066730 msgr2=0x7f487c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f487c066730 0x7f487c068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48900090b0 tx=0x7f489001c040 comp rx=0 tx=0).stop 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a00cf7a0 msgr2=0x7f48a014c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a00cf7a0 0x7f48a014c770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f488800c760 tx=0x7f488800ca70 comp rx=0 tx=0).stop 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 shutdown_connections 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f487c066730 0x7f487c068bf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a0159110 0x7f48a014ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a00cf7a0 0x7f48a014c770 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 --2- 172.21.15.3:0/839905054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a00b4e70 0x7f48a01524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 >> 172.21.15.3:0/839905054 conn(0x7f48a00b2d50 msgr2=0x7f48a00ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 shutdown_connections 2024-09-13T16:51:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.060+0000 7f48a5800700 1 -- 172.21.15.3:0/839905054 wait complete. 2024-09-13T16:51:23.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:23.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.260+0000 7f58edc00700 1 Processor -- start 2024-09-13T16:51:23.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.260+0000 7f58edc00700 1 -- start start 2024-09-13T16:51:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58edc00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58e81650f0 0x7f58e816bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58edc00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e80cc050 con 0x7f58e80cde70 2024-09-13T16:51:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58edc00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e80cc330 con 0x7f58e80ce250 2024-09-13T16:51:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58edc00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e80b2c70 con 0x7f58e81650f0 2024-09-13T16:51:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e7e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f58e80cde70 0x7f58e80cb7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56864/0 (socket says 172.21.15.3:56864) 2024-09-13T16:51:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e7e00700 1 -- 172.21.15.3:0/1106887019 learned_addr learned my addr 172.21.15.3:0/1106887019 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58ed200700 1 --2- 172.21.15.3:0/1106887019 >> v2:172.21.15.157:3300/0 conn(0x7f58e81650f0 0x7f58e816bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921842884 0 0) 0x7f58e80cc050 con 0x7f58e80cde70 2024-09-13T16:51:23.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58cc0035d0 con 0x7f58e80cde70 2024-09-13T16:51:23.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284109369 0 0) 0x7f58e80cc330 con 0x7f58e80ce250 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e80cc050 con 0x7f58e80ce250 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2929728705 0 0) 0x7f58cc0035d0 con 0x7f58e80cde70 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 >> v2:172.21.15.157:3300/0 conn(0x7f58e81650f0 msgr2=0x7f58e816bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 --2- 172.21.15.3:0/1106887019 >> v2:172.21.15.157:3300/0 conn(0x7f58e81650f0 0x7f58e816bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 >> v1:172.21.15.157:6789/0 conn(0x7f58e80ce250 legacy=0x7f58e81649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80cc610 con 0x7f58e80cde70 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f58d8001670 con 0x7f58e80cde70 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 >> v1:172.21.15.3:6789/0 conn(0x7f58e80cde70 legacy=0x7f58e80cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58cc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc004870 0x7f58cc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58cc005230 0x7f58cc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80b2c70 con 0x7f58cc003bb0 2024-09-13T16:51:23.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc00c0b0 con 0x7f58cc004870 2024-09-13T16:51:23.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc00c270 con 0x7f58cc005230 2024-09-13T16:51:23.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.264+0000 7f58e7e00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58cc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e7400700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc004870 0x7f58cc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58ed200700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58cc005230 0x7f58cc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e7e00700 1 -- 172.21.15.3:0/1106887019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58cc005230 msgr2=0x7f58cc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e7e00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58cc005230 0x7f58cc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e7e00700 1 -- 172.21.15.3:0/1106887019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc004870 msgr2=0x7f58cc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e7e00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc004870 0x7f58cc004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e7e00700 1 -- 172.21.15.3:0/1106887019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc0035d0 con 0x7f58cc003bb0 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58ed200700 1 -- 172.21.15.3:0/1106887019 reap_dead start 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e7e00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58cc003fb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f58d800c210 tx=0x7f58cc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b436552be8ec32 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d800cdb0 con 0x7f58cc003bb0 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d8005980 con 0x7f58cc003bb0 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58e6a00700 1 -- 172.21.15.3:0/1106887019 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8005d30 con 0x7f58cc003bb0 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58edc00700 1 -- 172.21.15.3:0/1106887019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 msgr2=0x7f58cc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58edc00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58cc003fb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f58d800c210 tx=0x7f58cc0058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58edc00700 1 -- 172.21.15.3:0/1106887019 shutdown_connections 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58edc00700 1 --2- 172.21.15.3:0/1106887019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58cc003fb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58edc00700 1 -- 172.21.15.3:0/1106887019 >> 172.21.15.3:0/1106887019 conn(0x7f58e80af620 msgr2=0x7f58e80afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58edc00700 1 -- 172.21.15.3:0/1106887019 shutdown_connections 2024-09-13T16:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.268+0000 7f58edc00700 1 -- 172.21.15.3:0/1106887019 wait complete. 2024-09-13T16:51:23.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58edc00700 1 Processor -- start 2024-09-13T16:51:23.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58edc00700 1 -- start start 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58edc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58e81f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58edc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e80ab2b0 0x7f58e8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58edc00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e80ad8a0 0x7f58e8160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58edc00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80ab6f0 con 0x7f58cc003bb0 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58edc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80ab900 con 0x7f58e80ab2b0 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58edc00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80abbe0 con 0x7f58e80ad8a0 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58e7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58e81f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58e7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58e81f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41460/0 (socket says 172.21.15.3:41460) 2024-09-13T16:51:23.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58e7e00700 1 -- 172.21.15.3:0/2247385351 learned_addr learned my addr 172.21.15.3:0/2247385351 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:23.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58e7400700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e80ab2b0 0x7f58e8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.272+0000 7f58ed200700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e80ad8a0 0x7f58e8160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58e7e00700 1 -- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e80ad8a0 msgr2=0x7f58e8160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58e7e00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e80ad8a0 0x7f58e8160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58e7e00700 1 -- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e80ab2b0 msgr2=0x7f58e8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58e7e00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e80ab2b0 0x7f58e8157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58e7e00700 1 -- 172.21.15.3:0/2247385351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80abec0 con 0x7f58cc003bb0 2024-09-13T16:51:23.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58e7e00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58e81f7d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c080 tx=0x7f58d8001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:23.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58dfe00700 1 -- 172.21.15.3:0/2247385351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8007bb0 con 0x7f58cc003bb0 2024-09-13T16:51:23.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58dfe00700 1 -- 172.21.15.3:0/2247385351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d8004890 con 0x7f58cc003bb0 2024-09-13T16:51:23.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58dfe00700 1 -- 172.21.15.3:0/2247385351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8004c00 con 0x7f58cc003bb0 2024-09-13T16:51:23.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e80ac1a0 con 0x7f58cc003bb0 2024-09-13T16:51:23.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e8160790 con 0x7f58cc003bb0 2024-09-13T16:51:23.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.276+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e80ac3c0 con 0x7f58cc003bb0 2024-09-13T16:51:23.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.280+0000 7f58dfe00700 1 -- 172.21.15.3:0/2247385351 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58d8007070 con 0x7f58cc003bb0 2024-09-13T16:51:23.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.280+0000 7f58dfe00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f58c8066760 0x7f58c8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.280+0000 7f58dfe00700 1 -- 172.21.15.3:0/2247385351 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f58d8084b60 con 0x7f58cc003bb0 2024-09-13T16:51:23.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.280+0000 7f58e7400700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f58c8066760 0x7f58c8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.284+0000 7f58e7400700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f58c8066760 0x7f58c8068c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f58e80cd3f0 tx=0x7f58d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:23.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.288+0000 7f58dfe00700 1 -- 172.21.15.3:0/2247385351 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58d8051cd0 con 0x7f58cc003bb0 2024-09-13T16:51:23.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.472+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58e80cc330 con 0x7f58cc003bb0 2024-09-13T16:51:23.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58dfe00700 1 -- 172.21.15.3:0/2247385351 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58d8024020 con 0x7f58cc003bb0 2024-09-13T16:51:23.480 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f58c8066760 msgr2=0x7f58c8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f58c8066760 0x7f58c8068c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f58e80cd3f0 tx=0x7f58d4009040 comp rx=0 tx=0).stop 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 msgr2=0x7f58e81f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58e81f7d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c080 tx=0x7f58d8001980 comp rx=0 tx=0).stop 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 shutdown_connections 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f58c8066760 0x7f58c8068c20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e80ad8a0 0x7f58e8160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e80ab2b0 0x7f58e8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 --2- 172.21.15.3:0/2247385351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58cc003bb0 0x7f58e81f7d40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 >> 172.21.15.3:0/2247385351 conn(0x7f58e80af620 msgr2=0x7f58e815ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:23.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 shutdown_connections 2024-09-13T16:51:23.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.476+0000 7f58edc00700 1 -- 172.21.15.3:0/2247385351 wait complete. 2024-09-13T16:51:23.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:23.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.676+0000 7fd132200700 1 Processor -- start 2024-09-13T16:51:23.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.676+0000 7fd132200700 1 -- start start 2024-09-13T16:51:23.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.676+0000 7fd132200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd12c1507b0 0x7fd12c159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd12c0b2ce0 0x7fd12c06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c06c650 0x7fd12c06a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c06e180 con 0x7fd12c0b2ce0 2024-09-13T16:51:23.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c06cba0 con 0x7fd12c1507b0 2024-09-13T16:51:23.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c06ce80 con 0x7fd12c06c650 2024-09-13T16:51:23.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd12c0b2ce0 0x7fd12c06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd12c0b2ce0 0x7fd12c06dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58394/0 (socket says 172.21.15.3:58394) 2024-09-13T16:51:23.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 -- 172.21.15.3:0/1376899469 learned_addr learned my addr 172.21.15.3:0/1376899469 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:23.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd131800700 1 --2- 172.21.15.3:0/1376899469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c06c650 0x7fd12c06a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd130e00700 1 --2- 172.21.15.3:0/1376899469 >> v2:172.21.15.157:3300/0 conn(0x7fd12c1507b0 0x7fd12c159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 -- 172.21.15.3:0/1376899469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c06c650 msgr2=0x7fd12c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:23.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 --2- 172.21.15.3:0/1376899469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c06c650 0x7fd12c06a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 -- 172.21.15.3:0/1376899469 >> v2:172.21.15.157:3300/0 conn(0x7fd12c1507b0 msgr2=0x7fd12c159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:23.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 --2- 172.21.15.3:0/1376899469 >> v2:172.21.15.157:3300/0 conn(0x7fd12c1507b0 0x7fd12c159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 -- 172.21.15.3:0/1376899469 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c06d160 con 0x7fd12c0b2ce0 2024-09-13T16:51:23.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd131800700 1 --2- 172.21.15.3:0/1376899469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c06c650 0x7fd12c06a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:23.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd130e00700 1 --2- 172.21.15.3:0/1376899469 >> v2:172.21.15.157:3300/0 conn(0x7fd12c1507b0 0x7fd12c159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:23.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12be00700 1 --2- 172.21.15.3:0/1376899469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0b2ce0 0x7fd12c06dc40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0b1880 tx=0x7fd118009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3df8c542370f619 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:23.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12b400700 1 -- 172.21.15.3:0/1376899469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800b8a0 con 0x7fd12c0b2ce0 2024-09-13T16:51:23.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12b400700 1 -- 172.21.15.3:0/1376899469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800ba00 con 0x7fd12c0b2ce0 2024-09-13T16:51:23.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd12b400700 1 -- 172.21.15.3:0/1376899469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800bd90 con 0x7fd12c0b2ce0 2024-09-13T16:51:23.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 -- 172.21.15.3:0/1376899469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0b2ce0 msgr2=0x7fd12c06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 --2- 172.21.15.3:0/1376899469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0b2ce0 0x7fd12c06dc40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0b1880 tx=0x7fd118009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 -- 172.21.15.3:0/1376899469 shutdown_connections 2024-09-13T16:51:23.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 --2- 172.21.15.3:0/1376899469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0b2ce0 0x7fd12c06dc40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 --2- 172.21.15.3:0/1376899469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c06c650 0x7fd12c06a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 --2- 172.21.15.3:0/1376899469 >> v2:172.21.15.157:3300/0 conn(0x7fd12c1507b0 0x7fd12c159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 -- 172.21.15.3:0/1376899469 >> 172.21.15.3:0/1376899469 conn(0x7fd12c0af620 msgr2=0x7fd12c0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:23.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.680+0000 7fd132200700 1 -- 172.21.15.3:0/1376899469 shutdown_connections 2024-09-13T16:51:23.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.684+0000 7fd132200700 1 -- 172.21.15.3:0/1376899469 wait complete. 2024-09-13T16:51:23.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.684+0000 7fd132200700 1 Processor -- start 2024-09-13T16:51:23.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.684+0000 7fd132200700 1 -- start start 2024-09-13T16:51:23.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd132200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c06c650 0x7fd12c151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd132200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2ce0 0x7fd12c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd132200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c1507b0 0x7fd12c1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd132200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c147580 con 0x7fd12c06c650 2024-09-13T16:51:23.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd132200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c154060 con 0x7fd12c1507b0 2024-09-13T16:51:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd132200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c147790 con 0x7fd12c0b2ce0 2024-09-13T16:51:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c06c650 0x7fd12c151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c06c650 0x7fd12c151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58406/0 (socket says 172.21.15.3:58406) 2024-09-13T16:51:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 -- 172.21.15.3:0/2894511489 learned_addr learned my addr 172.21.15.3:0/2894511489 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd12be00700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2ce0 0x7fd12c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd131800700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c1507b0 0x7fd12c1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 -- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2ce0 msgr2=0x7fd12c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2ce0 0x7fd12c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 -- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c1507b0 msgr2=0x7fd12c1592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c1507b0 0x7fd12c1592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 -- 172.21.15.3:0/2894511489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd118009570 con 0x7fd12c06c650 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd131800700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c1507b0 0x7fd12c1592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd130e00700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c06c650 0x7fd12c151790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd12000c7c0 tx=0x7fd12000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd128c00700 1 -- 172.21.15.3:0/2894511489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120011800 con 0x7fd12c06c650 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd128c00700 1 -- 172.21.15.3:0/2894511489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd120011960 con 0x7fd12c06c650 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c147a70 con 0x7fd12c06c650 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd128c00700 1 -- 172.21.15.3:0/2894511489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120011cf0 con 0x7fd12c06c650 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.688+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c148030 con 0x7fd12c06c650 2024-09-13T16:51:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.692+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c06dc40 con 0x7fd12c06c650 2024-09-13T16:51:23.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.692+0000 7fd128c00700 1 -- 172.21.15.3:0/2894511489 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd120021350 con 0x7fd12c06c650 2024-09-13T16:51:23.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.692+0000 7fd128c00700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd10c066730 0x7fd10c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:23.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.692+0000 7fd128c00700 1 -- 172.21.15.3:0/2894511489 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd120084520 con 0x7fd12c06c650 2024-09-13T16:51:23.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.696+0000 7fd12be00700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd10c066730 0x7fd10c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:23.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.696+0000 7fd12be00700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd10c066730 0x7fd10c068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd118000c00 tx=0x7fd118008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.700+0000 7fd128c00700 1 -- 172.21.15.3:0/2894511489 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd120051440 con 0x7fd12c06c650 2024-09-13T16:51:23.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.884+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd12c0b4a60 con 0x7fd12c06c650 2024-09-13T16:51:23.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.884+0000 7fd128c00700 1 -- 172.21.15.3:0/2894511489 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd120054a60 con 0x7fd12c06c650 2024-09-13T16:51:23.890 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd10c066730 msgr2=0x7fd10c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd10c066730 0x7fd10c068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd118000c00 tx=0x7fd118008030 comp rx=0 tx=0).stop 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c06c650 msgr2=0x7fd12c151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c06c650 0x7fd12c151790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd12000c7c0 tx=0x7fd12000cb80 comp rx=0 tx=0).stop 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 shutdown_connections 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd10c066730 0x7fd10c068bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c1507b0 0x7fd12c1592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2ce0 0x7fd12c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 --2- 172.21.15.3:0/2894511489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c06c650 0x7fd12c151790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 >> 172.21.15.3:0/2894511489 conn(0x7fd12c0af620 msgr2=0x7fd12c0b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 shutdown_connections 2024-09-13T16:51:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:23.888+0000 7fd132200700 1 -- 172.21.15.3:0/2894511489 wait complete. 2024-09-13T16:51:23.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.084+0000 7fa0db200700 1 Processor -- start 2024-09-13T16:51:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.084+0000 7fa0db200700 1 -- start start 2024-09-13T16:51:24.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0db200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0d4159d30 0x7fa0d415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0db200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d40721a0 0x7fa0d415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0db200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d40cdda0 con 0x7fa0d4159950 2024-09-13T16:51:24.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0db200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d40ce990 con 0x7fa0d4159d30 2024-09-13T16:51:24.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0db200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40cdff0 con 0x7fa0d40721a0 2024-09-13T16:51:24.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0d3e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa0d4159950 0x7fa0d40b3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:46720/0 (socket says 172.21.15.3:46720) 2024-09-13T16:51:24.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0d3e00700 1 -- 172.21.15.3:0/1332590489 learned_addr learned my addr 172.21.15.3:0/1332590489 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0cb400700 1 --2- 172.21.15.3:0/1332590489 >> v2:172.21.15.157:3300/0 conn(0x7fa0d4159d30 0x7fa0d415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0d9200700 1 --2- 172.21.15.3:0/1332590489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d40721a0 0x7fa0d415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.088+0000 7fa0d3400700 1 -- 172.21.15.3:0/1332590489 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808940102 0 0) 0x7fa0d40cdda0 con 0x7fa0d4159950 2024-09-13T16:51:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0d3400700 1 -- 172.21.15.3:0/1332590489 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b80035d0 con 0x7fa0d4159950 2024-09-13T16:51:24.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0cb400700 1 -- 172.21.15.3:0/1332590489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d40721a0 msgr2=0x7fa0d415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:24.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0cb400700 1 --2- 172.21.15.3:0/1332590489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d40721a0 0x7fa0d415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0cb400700 1 -- 172.21.15.3:0/1332590489 >> v1:172.21.15.157:6789/0 conn(0x7fa0d4159950 legacy=0x7fa0d40b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0cb400700 1 -- 172.21.15.3:0/1332590489 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40ce2d0 con 0x7fa0d4159d30 2024-09-13T16:51:24.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0cb400700 1 --2- 172.21.15.3:0/1332590489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d4159d30 0x7fa0d415d6d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009850 tx=0x7fa0bc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eff050095486dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0d3400700 1 -- 172.21.15.3:0/1332590489 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc011830 con 0x7fa0d4159d30 2024-09-13T16:51:24.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0d3400700 1 -- 172.21.15.3:0/1332590489 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc011990 con 0x7fa0d4159d30 2024-09-13T16:51:24.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0d3400700 1 -- 172.21.15.3:0/1332590489 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc011d40 con 0x7fa0d4159d30 2024-09-13T16:51:24.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0db200700 1 -- 172.21.15.3:0/1332590489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d4159d30 msgr2=0x7fa0d415d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0db200700 1 --2- 172.21.15.3:0/1332590489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d4159d30 0x7fa0d415d6d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009850 tx=0x7fa0bc009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:24.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0db200700 1 -- 172.21.15.3:0/1332590489 shutdown_connections 2024-09-13T16:51:24.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0db200700 1 --2- 172.21.15.3:0/1332590489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d40721a0 0x7fa0d415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0db200700 1 --2- 172.21.15.3:0/1332590489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d4159d30 0x7fa0d415d6d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0db200700 1 -- 172.21.15.3:0/1332590489 >> 172.21.15.3:0/1332590489 conn(0x7fa0d406a390 msgr2=0x7fa0d406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:24.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0db200700 1 -- 172.21.15.3:0/1332590489 shutdown_connections 2024-09-13T16:51:24.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.092+0000 7fa0db200700 1 -- 172.21.15.3:0/1332590489 wait complete. 2024-09-13T16:51:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.096+0000 7fa0db200700 1 Processor -- start 2024-09-13T16:51:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.096+0000 7fa0db200700 1 -- start start 2024-09-13T16:51:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0db200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d40721a0 0x7fa0d4147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0db200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0d4159950 0x7fa0d41446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0db200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d4159d30 0x7fa0d406a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0db200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d4148cf0 con 0x7fa0d4159950 2024-09-13T16:51:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0db200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d4147e40 con 0x7fa0d40721a0 2024-09-13T16:51:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0db200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d41480e0 con 0x7fa0d4159d30 2024-09-13T16:51:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0d4159950 0x7fa0d41446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0d4159950 0x7fa0d41446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58422/0 (socket says 172.21.15.3:58422) 2024-09-13T16:51:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 -- 172.21.15.3:0/2756662992 learned_addr learned my addr 172.21.15.3:0/2756662992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0d9200700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d4159d30 0x7fa0d406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0d3e00700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d40721a0 0x7fa0d4147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 -- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d4159d30 msgr2=0x7fa0d406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d4159d30 0x7fa0d406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 -- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d40721a0 msgr2=0x7fa0d4147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d40721a0 0x7fa0d4147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 -- 172.21.15.3:0/2756662992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0094b0 con 0x7fa0d4159950 2024-09-13T16:51:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0d3e00700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d40721a0 0x7fa0d4147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0cb400700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0d4159950 0x7fa0d41446e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc014040 tx=0x7fa0bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0d0c00700 1 -- 172.21.15.3:0/2756662992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0036c0 con 0x7fa0d4159950 2024-09-13T16:51:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0d0c00700 1 -- 172.21.15.3:0/2756662992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc01e070 con 0x7fa0d4159950 2024-09-13T16:51:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d41483c0 con 0x7fa0d4159950 2024-09-13T16:51:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0d0c00700 1 -- 172.21.15.3:0/2756662992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc003a80 con 0x7fa0d4159950 2024-09-13T16:51:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.100+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d4148980 con 0x7fa0d4159950 2024-09-13T16:51:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.104+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d40b3ad0 con 0x7fa0d4159950 2024-09-13T16:51:24.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.104+0000 7fa0d0c00700 1 -- 172.21.15.3:0/2756662992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa0bc003be0 con 0x7fa0d4159950 2024-09-13T16:51:24.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.108+0000 7fa0d0c00700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0b0066730 0x7fa0b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.108+0000 7fa0d0c00700 1 -- 172.21.15.3:0/2756662992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa0bc020070 con 0x7fa0d4159950 2024-09-13T16:51:24.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.108+0000 7fa0d3e00700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0b0066730 0x7fa0b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.108+0000 7fa0d3e00700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0b0066730 0x7fa0b0068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4001560 tx=0x7fa0c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.112+0000 7fa0d0c00700 1 -- 172.21.15.3:0/2756662992 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa0bc052960 con 0x7fa0d4159950 2024-09-13T16:51:24.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.300+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0d406cf90 con 0x7fa0d4159950 2024-09-13T16:51:24.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.304+0000 7fa0d0c00700 1 -- 172.21.15.3:0/2756662992 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7fa0bc006ce0 con 0x7fa0d4159950 2024-09-13T16:51:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:24.309 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.304+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0b0066730 msgr2=0x7fa0b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.304+0000 7fa0db200700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0b0066730 0x7fa0b0068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4001560 tx=0x7fa0c4009040 comp rx=0 tx=0).stop 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.304+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0d4159950 msgr2=0x7fa0d41446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.304+0000 7fa0db200700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0d4159950 0x7fa0d41446e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc014040 tx=0x7fa0bc004770 comp rx=0 tx=0).stop 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.308+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 shutdown_connections 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.308+0000 7fa0db200700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa0b0066730 0x7fa0b0068bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.308+0000 7fa0db200700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d4159d30 0x7fa0d406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.308+0000 7fa0db200700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0d4159950 0x7fa0d41446e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.308+0000 7fa0db200700 1 --2- 172.21.15.3:0/2756662992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d40721a0 0x7fa0d4147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.308+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 >> 172.21.15.3:0/2756662992 conn(0x7fa0d406a390 msgr2=0x7fa0d406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.308+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 shutdown_connections 2024-09-13T16:51:24.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.308+0000 7fa0db200700 1 -- 172.21.15.3:0/2756662992 wait complete. 2024-09-13T16:51:24.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:24.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.504+0000 7f6dd9e00700 1 Processor -- start 2024-09-13T16:51:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.504+0000 7f6dd9e00700 1 -- start start 2024-09-13T16:51:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd9e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6dd40cde70 0x7f6dd40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd9e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6dd40cb7c0 0x7f6dd41649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd9e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd4164f20 0x7f6dd4161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd9e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd40b2c70 con 0x7f6dd40cde70 2024-09-13T16:51:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd9e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd40ce790 con 0x7f6dd40cb7c0 2024-09-13T16:51:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd9e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd4165390 con 0x7f6dd4164f20 2024-09-13T16:51:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6dd40cde70 0x7f6dd40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6dd40cde70 0x7f6dd40ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58434/0 (socket says 172.21.15.3:58434) 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 -- 172.21.15.3:0/911896705 learned_addr learned my addr 172.21.15.3:0/911896705 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd9400700 1 --2- 172.21.15.3:0/911896705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd4164f20 0x7f6dd4161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd3e00700 1 --2- 172.21.15.3:0/911896705 >> v2:172.21.15.157:3300/0 conn(0x7f6dd40cb7c0 0x7f6dd41649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 -- 172.21.15.3:0/911896705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd4164f20 msgr2=0x7f6dd4161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 --2- 172.21.15.3:0/911896705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd4164f20 0x7f6dd4161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 -- 172.21.15.3:0/911896705 >> v2:172.21.15.157:3300/0 conn(0x7f6dd40cb7c0 msgr2=0x7f6dd41649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 --2- 172.21.15.3:0/911896705 >> v2:172.21.15.157:3300/0 conn(0x7f6dd40cb7c0 0x7f6dd41649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 -- 172.21.15.3:0/911896705 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd40cc050 con 0x7f6dd40cde70 2024-09-13T16:51:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.508+0000 7f6dd8a00700 1 --2- 172.21.15.3:0/911896705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40ce250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40b1830 tx=0x7f6dc8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f705a0f2c900aec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd3400700 1 -- 172.21.15.3:0/911896705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8011830 con 0x7f6dd40cde70 2024-09-13T16:51:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd3400700 1 -- 172.21.15.3:0/911896705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc8011990 con 0x7f6dd40cde70 2024-09-13T16:51:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd3400700 1 -- 172.21.15.3:0/911896705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8011d20 con 0x7f6dd40cde70 2024-09-13T16:51:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 -- 172.21.15.3:0/911896705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 msgr2=0x7f6dd40ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/911896705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40ce250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40b1830 tx=0x7f6dc8009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 -- 172.21.15.3:0/911896705 shutdown_connections 2024-09-13T16:51:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/911896705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd4164f20 0x7f6dd4161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/911896705 >> v2:172.21.15.157:3300/0 conn(0x7f6dd40cb7c0 0x7f6dd41649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/911896705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40ce250 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 -- 172.21.15.3:0/911896705 >> 172.21.15.3:0/911896705 conn(0x7f6dd40af620 msgr2=0x7f6dd40afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 -- 172.21.15.3:0/911896705 shutdown_connections 2024-09-13T16:51:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.512+0000 7f6dd9e00700 1 -- 172.21.15.3:0/911896705 wait complete. 2024-09-13T16:51:24.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9e00700 1 Processor -- start 2024-09-13T16:51:24.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9e00700 1 -- start start 2024-09-13T16:51:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd40cb7c0 0x7f6dd41ec660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd4164f20 0x7f6dd415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd41eccb0 con 0x7f6dd40cde70 2024-09-13T16:51:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd41ecf90 con 0x7f6dd4164f20 2024-09-13T16:51:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd41ed270 con 0x7f6dd40cb7c0 2024-09-13T16:51:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58436/0 (socket says 172.21.15.3:58436) 2024-09-13T16:51:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 -- 172.21.15.3:0/597978906 learned_addr learned my addr 172.21.15.3:0/597978906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd8a00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd40cb7c0 0x7f6dd41ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9400700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd4164f20 0x7f6dd415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 -- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd40cb7c0 msgr2=0x7f6dd41ec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd40cb7c0 0x7f6dd41ec660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 -- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd4164f20 msgr2=0x7f6dd415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd4164f20 0x7f6dd415a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 -- 172.21.15.3:0/597978906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8009570 con 0x7f6dd40cde70 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd9400700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd4164f20 0x7f6dd415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.516+0000 7f6dd3e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40b4310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000e7c0 tx=0x7f6dc000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.520+0000 7f6dd0c00700 1 -- 172.21.15.3:0/597978906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0013680 con 0x7f6dd40cde70 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.520+0000 7f6dd0c00700 1 -- 172.21.15.3:0/597978906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc00137e0 con 0x7f6dd40cde70 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.520+0000 7f6dd0c00700 1 -- 172.21.15.3:0/597978906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0013b90 con 0x7f6dd40cde70 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.520+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd41ed550 con 0x7f6dd40cde70 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.520+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd4154160 con 0x7f6dd40cde70 2024-09-13T16:51:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.520+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd41649e0 con 0x7f6dd40cde70 2024-09-13T16:51:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.520+0000 7f6dd0c00700 1 -- 172.21.15.3:0/597978906 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6dc0013cf0 con 0x7f6dd40cde70 2024-09-13T16:51:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.524+0000 7f6dd0c00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6db4066730 0x7f6db4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.524+0000 7f6dd0c00700 1 -- 172.21.15.3:0/597978906 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6dc0084540 con 0x7f6dd40cde70 2024-09-13T16:51:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.524+0000 7f6dd8a00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6db4066730 0x7f6db4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.524+0000 7f6dd8a00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6db4066730 0x7f6db4068bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8000c00 tx=0x7f6dc8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.532+0000 7f6dd0c00700 1 -- 172.21.15.3:0/597978906 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6dc0051450 con 0x7f6dd40cde70 2024-09-13T16:51:24.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.712+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6dd41591c0 con 0x7f6dd40cde70 2024-09-13T16:51:24.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd0c00700 1 -- 172.21.15.3:0/597978906 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6dc0059e60 con 0x7f6dd40cde70 2024-09-13T16:51:24.721 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6db4066730 msgr2=0x7f6db4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6db4066730 0x7f6db4068bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8000c00 tx=0x7f6dc8008030 comp rx=0 tx=0).stop 2024-09-13T16:51:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 msgr2=0x7f6dd40b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40b4310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000e7c0 tx=0x7f6dc000eb80 comp rx=0 tx=0).stop 2024-09-13T16:51:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 shutdown_connections 2024-09-13T16:51:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6db4066730 0x7f6db4068bf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd4164f20 0x7f6dd415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd40cde70 0x7f6dd40b4310 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 --2- 172.21.15.3:0/597978906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd40cb7c0 0x7f6dd41ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.716+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 >> 172.21.15.3:0/597978906 conn(0x7f6dd40af620 msgr2=0x7f6dd4161280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.720+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 shutdown_connections 2024-09-13T16:51:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.720+0000 7f6dd9e00700 1 -- 172.21.15.3:0/597978906 wait complete. 2024-09-13T16:51:24.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:24.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.928+0000 7fc8da600700 1 Processor -- start 2024-09-13T16:51:24.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.928+0000 7fc8da600700 1 -- start start 2024-09-13T16:51:24.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8da600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc8d415b1c0 0x7fc8d40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8da600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 0x7fc8d40ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8da600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d40cd1c0 con 0x7fc8d4158740 2024-09-13T16:51:24.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8da600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d40b3960 con 0x7fc8d415b1c0 2024-09-13T16:51:24.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8da600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40b1f30 con 0x7fc8d40cc830 2024-09-13T16:51:24.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc8d415b1c0 0x7fc8d40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc8d415b1c0 0x7fc8d40ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58440/0 (socket says 172.21.15.3:58440) 2024-09-13T16:51:24.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2a00700 1 -- 172.21.15.3:0/739833513 learned_addr learned my addr 172.21.15.3:0/739833513 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:24.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d3e00700 1 --2- 172.21.15.3:0/739833513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 0x7fc8d40ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2a00700 1 -- 172.21.15.3:0/739833513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 msgr2=0x7fc8d40ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:24.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2a00700 1 --2- 172.21.15.3:0/739833513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 0x7fc8d40ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2a00700 1 -- 172.21.15.3:0/739833513 >> v1:172.21.15.157:6789/0 conn(0x7fc8d4158740 legacy=0x7fc8d40b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2a00700 1 -- 172.21.15.3:0/739833513 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d415b710 con 0x7fc8d415b1c0 2024-09-13T16:51:24.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2a00700 1 --2- 172.21.15.3:0/739833513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40ce6e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009910 tx=0x7fc8c0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd19eb75ecff47dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2000700 1 -- 172.21.15.3:0/739833513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000ba60 con 0x7fc8d415b1c0 2024-09-13T16:51:24.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2000700 1 -- 172.21.15.3:0/739833513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c000bbc0 con 0x7fc8d415b1c0 2024-09-13T16:51:24.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.932+0000 7fc8d2000700 1 -- 172.21.15.3:0/739833513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0009e60 con 0x7fc8d415b1c0 2024-09-13T16:51:24.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 -- 172.21.15.3:0/739833513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 msgr2=0x7fc8d40ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 --2- 172.21.15.3:0/739833513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40ce6e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009910 tx=0x7fc8c0009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:24.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 -- 172.21.15.3:0/739833513 shutdown_connections 2024-09-13T16:51:24.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 --2- 172.21.15.3:0/739833513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 0x7fc8d40ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 --2- 172.21.15.3:0/739833513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40ce6e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 -- 172.21.15.3:0/739833513 >> 172.21.15.3:0/739833513 conn(0x7fc8d406b280 msgr2=0x7fc8d406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:24.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 -- 172.21.15.3:0/739833513 shutdown_connections 2024-09-13T16:51:24.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 -- 172.21.15.3:0/739833513 wait complete. 2024-09-13T16:51:24.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.936+0000 7fc8da600700 1 Processor -- start 2024-09-13T16:51:24.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8da600700 1 -- start start 2024-09-13T16:51:24.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8da600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 0x7fc8d414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8da600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d4158740 0x7fc8d4149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8da600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8da600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40b0210 con 0x7fc8d415b1c0 2024-09-13T16:51:24.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8da600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d4149f70 con 0x7fc8d4158740 2024-09-13T16:51:24.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8da600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40aff30 con 0x7fc8d40cc830 2024-09-13T16:51:24.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d4158740 0x7fc8d4149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58444/0 (socket says 172.21.15.3:58444) 2024-09-13T16:51:24.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3e00700 1 -- 172.21.15.3:0/3065353315 learned_addr learned my addr 172.21.15.3:0/3065353315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:24.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3400700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 0x7fc8d414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3e00700 1 -- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 msgr2=0x7fc8d414a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3e00700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 0x7fc8d414a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3e00700 1 -- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d4158740 msgr2=0x7fc8d4149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:24.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3e00700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d4158740 0x7fc8d4149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:24.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d3e00700 1 -- 172.21.15.3:0/3065353315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c0009570 con 0x7fc8d415b1c0 2024-09-13T16:51:24.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.940+0000 7fc8d2a00700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d4158740 0x7fc8d4149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.944+0000 7fc8d3e00700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40af470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc007e60 tx=0x7fc8cc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.944+0000 7fc8cb400700 1 -- 172.21.15.3:0/3065353315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00ed30 con 0x7fc8d415b1c0 2024-09-13T16:51:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.944+0000 7fc8cb400700 1 -- 172.21.15.3:0/3065353315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc014420 con 0x7fc8d415b1c0 2024-09-13T16:51:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.944+0000 7fc8cb400700 1 -- 172.21.15.3:0/3065353315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0147d0 con 0x7fc8d415b1c0 2024-09-13T16:51:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.944+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d40aca50 con 0x7fc8d415b1c0 2024-09-13T16:51:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.944+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d40ad010 con 0x7fc8d415b1c0 2024-09-13T16:51:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.944+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d40ac8b0 con 0x7fc8d415b1c0 2024-09-13T16:51:24.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.944+0000 7fc8cb400700 1 -- 172.21.15.3:0/3065353315 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8cc014930 con 0x7fc8d415b1c0 2024-09-13T16:51:24.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.948+0000 7fc8cb400700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8b4066730 0x7fc8b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:24.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.948+0000 7fc8cb400700 1 -- 172.21.15.3:0/3065353315 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc8cc084b10 con 0x7fc8d415b1c0 2024-09-13T16:51:24.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.948+0000 7fc8d3400700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8b4066730 0x7fc8b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:24.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.948+0000 7fc8d3400700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8b4066730 0x7fc8b4068bf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4000f60 tx=0x7fc8c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:24.956+0000 7fc8cb400700 1 -- 172.21.15.3:0/3065353315 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8d40ac8b0 con 0x7fc8d415b1c0 2024-09-13T16:51:25.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.136+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8d40ac8b0 con 0x7fc8d415b1c0 2024-09-13T16:51:25.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.136+0000 7fc8cb400700 1 -- 172.21.15.3:0/3065353315 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8d40ac8b0 con 0x7fc8d415b1c0 2024-09-13T16:51:25.142 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:25.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8b4066730 msgr2=0x7fc8b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8b4066730 0x7fc8b4068bf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4000f60 tx=0x7fc8c4009040 comp rx=0 tx=0).stop 2024-09-13T16:51:25.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 msgr2=0x7fc8d40af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40af470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc007e60 tx=0x7fc8cc00e3c0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 shutdown_connections 2024-09-13T16:51:25.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8b4066730 0x7fc8b4068bf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8d415b1c0 0x7fc8d40af470 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d4158740 0x7fc8d4149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 --2- 172.21.15.3:0/3065353315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d40cc830 0x7fc8d414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 >> 172.21.15.3:0/3065353315 conn(0x7fc8d406b280 msgr2=0x7fc8d41594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:25.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 shutdown_connections 2024-09-13T16:51:25.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.140+0000 7fc8da600700 1 -- 172.21.15.3:0/3065353315 wait complete. 2024-09-13T16:51:25.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.344+0000 7fa791400700 1 Processor -- start 2024-09-13T16:51:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa791400700 1 -- start start 2024-09-13T16:51:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa791400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa78c0d10d0 0x7fa78c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa791400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0b55f0 con 0x7fa78c15b260 2024-09-13T16:51:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa791400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0b58d0 con 0x7fa78c15a940 2024-09-13T16:51:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa791400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78c0b17f0 con 0x7fa78c0d10d0 2024-09-13T16:51:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa783e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa78c0d10d0 0x7fa78c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa78b200700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa78c15b260 0x7fa78c0b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51528/0 (socket says 172.21.15.3:51528) 2024-09-13T16:51:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa78b200700 1 -- 172.21.15.3:0/796868514 learned_addr learned my addr 172.21.15.3:0/796868514 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa78a800700 1 --1- 172.21.15.3:0/796868514 >> v1:172.21.15.157:6789/0 conn(0x7fa78c15a940 0x7fa78c0d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:46740/0 (socket says 172.21.15.3:46740) 2024-09-13T16:51:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.348+0000 7fa789e00700 1 -- 172.21.15.3:0/796868514 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120558813 0 0) 0x7fa78c0b55f0 con 0x7fa78c15b260 2024-09-13T16:51:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa789e00700 1 -- 172.21.15.3:0/796868514 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7700035d0 con 0x7fa78c15b260 2024-09-13T16:51:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa789e00700 1 -- 172.21.15.3:0/796868514 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288131633 0 0) 0x7fa78c0b58d0 con 0x7fa78c15a940 2024-09-13T16:51:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa783e00700 1 -- 172.21.15.3:0/796868514 >> v1:172.21.15.157:6789/0 conn(0x7fa78c15a940 legacy=0x7fa78c0d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa783e00700 1 -- 172.21.15.3:0/796868514 >> v1:172.21.15.3:6789/0 conn(0x7fa78c15b260 legacy=0x7fa78c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa783e00700 1 -- 172.21.15.3:0/796868514 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0b5bb0 con 0x7fa78c0d10d0 2024-09-13T16:51:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa783e00700 1 --2- 172.21.15.3:0/796868514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c0d10d0 0x7fa78c15ad20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa774009850 tx=0x7fa774009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4469f91501190bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa789e00700 1 -- 172.21.15.3:0/796868514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774011830 con 0x7fa78c0d10d0 2024-09-13T16:51:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa789e00700 1 -- 172.21.15.3:0/796868514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa774011990 con 0x7fa78c0d10d0 2024-09-13T16:51:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa789e00700 1 -- 172.21.15.3:0/796868514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774011d00 con 0x7fa78c0d10d0 2024-09-13T16:51:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa791400700 1 -- 172.21.15.3:0/796868514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c0d10d0 msgr2=0x7fa78c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa791400700 1 --2- 172.21.15.3:0/796868514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c0d10d0 0x7fa78c15ad20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa774009850 tx=0x7fa774009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa791400700 1 -- 172.21.15.3:0/796868514 shutdown_connections 2024-09-13T16:51:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa791400700 1 --2- 172.21.15.3:0/796868514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c0d10d0 0x7fa78c15ad20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa791400700 1 -- 172.21.15.3:0/796868514 >> 172.21.15.3:0/796868514 conn(0x7fa78c0b29f0 msgr2=0x7fa78c0b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa791400700 1 -- 172.21.15.3:0/796868514 shutdown_connections 2024-09-13T16:51:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.352+0000 7fa791400700 1 -- 172.21.15.3:0/796868514 wait complete. 2024-09-13T16:51:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa791400700 1 Processor -- start 2024-09-13T16:51:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa791400700 1 -- start start 2024-09-13T16:51:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa791400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78c0d10d0 0x7fa78c14e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa791400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78c15a940 0x7fa78c13c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa791400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c15b260 0x7fa78c13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa791400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c14dc10 con 0x7fa78c15b260 2024-09-13T16:51:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa791400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c13a4c0 con 0x7fa78c15a940 2024-09-13T16:51:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa791400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1508e0 con 0x7fa78c0d10d0 2024-09-13T16:51:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa78b200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c15b260 0x7fa78c13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa78a800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78c0d10d0 0x7fa78c14e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa78b200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c15b260 0x7fa78c13cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58462/0 (socket says 172.21.15.3:58462) 2024-09-13T16:51:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa78b200700 1 -- 172.21.15.3:0/3676479396 learned_addr learned my addr 172.21.15.3:0/3676479396 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.356+0000 7fa783e00700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78c15a940 0x7fa78c13c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa78b200700 1 -- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78c0d10d0 msgr2=0x7fa78c14e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa78b200700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78c0d10d0 0x7fa78c14e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa78b200700 1 -- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78c15a940 msgr2=0x7fa78c13c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa78b200700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78c15a940 0x7fa78c13c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa78b200700 1 -- 172.21.15.3:0/3676479396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740094b0 con 0x7fa78c15b260 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa78b200700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c15b260 0x7fa78c13cd70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa78400ca20 tx=0x7fa78400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa782a00700 1 -- 172.21.15.3:0/3676479396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784007480 con 0x7fa78c15b260 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa782a00700 1 -- 172.21.15.3:0/3676479396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7840075e0 con 0x7fa78c15b260 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c150bc0 con 0x7fa78c15b260 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa782a00700 1 -- 172.21.15.3:0/3676479396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784007970 con 0x7fa78c15b260 2024-09-13T16:51:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c151180 con 0x7fa78c15b260 2024-09-13T16:51:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c0b11e0 con 0x7fa78c15b260 2024-09-13T16:51:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.360+0000 7fa782a00700 1 -- 172.21.15.3:0/3676479396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa78401b3b0 con 0x7fa78c15b260 2024-09-13T16:51:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.364+0000 7fa782a00700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa768066730 0x7fa768068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.364+0000 7fa782a00700 1 -- 172.21.15.3:0/3676479396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa784083e40 con 0x7fa78c15b260 2024-09-13T16:51:25.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.364+0000 7fa78a800700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa768066730 0x7fa768068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.364+0000 7fa78a800700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa768066730 0x7fa768068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa77c000f60 tx=0x7fa77c007e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:25.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.368+0000 7fa782a00700 1 -- 172.21.15.3:0/3676479396 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa784050d50 con 0x7fa78c15b260 2024-09-13T16:51:25.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.548+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa78c15ad20 con 0x7fa78c15b260 2024-09-13T16:51:25.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.552+0000 7fa782a00700 1 -- 172.21.15.3:0/3676479396 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa784054370 con 0x7fa78c15b260 2024-09-13T16:51:25.557 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.552+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa768066730 msgr2=0x7fa768068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.552+0000 7fa791400700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa768066730 0x7fa768068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa77c000f60 tx=0x7fa77c007e00 comp rx=0 tx=0).stop 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.552+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c15b260 msgr2=0x7fa78c13cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.552+0000 7fa791400700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c15b260 0x7fa78c13cd70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa78400ca20 tx=0x7fa78400ca50 comp rx=0 tx=0).stop 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.556+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 shutdown_connections 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.556+0000 7fa791400700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa768066730 0x7fa768068bf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.556+0000 7fa791400700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa78c15b260 0x7fa78c13cd70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.556+0000 7fa791400700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78c15a940 0x7fa78c13c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.556+0000 7fa791400700 1 --2- 172.21.15.3:0/3676479396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78c0d10d0 0x7fa78c14e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.556+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 >> 172.21.15.3:0/3676479396 conn(0x7fa78c0b29f0 msgr2=0x7fa78c0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:25.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.556+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 shutdown_connections 2024-09-13T16:51:25.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.556+0000 7fa791400700 1 -- 172.21.15.3:0/3676479396 wait complete. 2024-09-13T16:51:25.583 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:25.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.760+0000 7f1d53200700 1 Processor -- start 2024-09-13T16:51:25.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.760+0000 7f1d53200700 1 -- start start 2024-09-13T16:51:25.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d4c0b3160 0x7f1d4c0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c0717f0 0x7f1d4c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c165cf0 con 0x7f1d4c1611a0 2024-09-13T16:51:25.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c071c80 con 0x7f1d4c0b3160 2024-09-13T16:51:25.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0b35a0 con 0x7f1d4c0717f0 2024-09-13T16:51:25.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d4b400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d4c0b3160 0x7f1d4c0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d51200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c0717f0 0x7f1d4c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d4b400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d4c0b3160 0x7f1d4c0712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58470/0 (socket says 172.21.15.3:58470) 2024-09-13T16:51:25.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d4b400700 1 -- 172.21.15.3:0/2095078783 learned_addr learned my addr 172.21.15.3:0/2095078783 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:25.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d4be00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1d4c1611a0 0x7f1d4c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:46752/0 (socket says 172.21.15.3:46752) 2024-09-13T16:51:25.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d51200700 1 -- 172.21.15.3:0/2095078783 >> v2:172.21.15.3:3300/0 conn(0x7f1d4c0b3160 msgr2=0x7f1d4c0712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:25.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d51200700 1 --2- 172.21.15.3:0/2095078783 >> v2:172.21.15.3:3300/0 conn(0x7f1d4c0b3160 0x7f1d4c0712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d51200700 1 -- 172.21.15.3:0/2095078783 >> v1:172.21.15.157:6789/0 conn(0x7f1d4c1611a0 legacy=0x7f1d4c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d51200700 1 -- 172.21.15.3:0/2095078783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c165170 con 0x7f1d4c0717f0 2024-09-13T16:51:25.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d51200700 1 --2- 172.21.15.3:0/2095078783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c0717f0 0x7f1d4c0b2280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009d90 tx=0x7f1d4400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58fd862c74f2f03e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:25.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d4aa00700 1 -- 172.21.15.3:0/2095078783 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400e070 con 0x7f1d4c0717f0 2024-09-13T16:51:25.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d4aa00700 1 -- 172.21.15.3:0/2095078783 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4400bb20 con 0x7f1d4c0717f0 2024-09-13T16:51:25.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d4aa00700 1 -- 172.21.15.3:0/2095078783 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44012430 con 0x7f1d4c0717f0 2024-09-13T16:51:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 -- 172.21.15.3:0/2095078783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c0717f0 msgr2=0x7f1d4c0b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 --2- 172.21.15.3:0/2095078783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c0717f0 0x7f1d4c0b2280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009d90 tx=0x7f1d4400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 -- 172.21.15.3:0/2095078783 shutdown_connections 2024-09-13T16:51:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 --2- 172.21.15.3:0/2095078783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c0717f0 0x7f1d4c0b2280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.764+0000 7f1d53200700 1 --2- 172.21.15.3:0/2095078783 >> v2:172.21.15.3:3300/0 conn(0x7f1d4c0b3160 0x7f1d4c0712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.768+0000 7f1d53200700 1 -- 172.21.15.3:0/2095078783 >> 172.21.15.3:0/2095078783 conn(0x7f1d4c06b280 msgr2=0x7f1d4c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.768+0000 7f1d53200700 1 -- 172.21.15.3:0/2095078783 shutdown_connections 2024-09-13T16:51:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.768+0000 7f1d53200700 1 -- 172.21.15.3:0/2095078783 wait complete. 2024-09-13T16:51:25.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.768+0000 7f1d53200700 1 Processor -- start 2024-09-13T16:51:25.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.768+0000 7f1d53200700 1 -- start start 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d53200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d4c0717f0 0x7f1d4c0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d53200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c0b3160 0x7f1d4c0ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d53200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c1611a0 0x7f1d4c0ab830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d53200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0cac10 con 0x7f1d4c0717f0 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d53200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0aa790 con 0x7f1d4c0b3160 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d53200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0aaa70 con 0x7f1d4c1611a0 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d4c0717f0 0x7f1d4c0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d4c0717f0 0x7f1d4c0ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58478/0 (socket says 172.21.15.3:58478) 2024-09-13T16:51:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 -- 172.21.15.3:0/1115231104 learned_addr learned my addr 172.21.15.3:0/1115231104 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4b400700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c0b3160 0x7f1d4c0ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d51200700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c1611a0 0x7f1d4c0ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 -- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c1611a0 msgr2=0x7f1d4c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c1611a0 0x7f1d4c0ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 -- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c0b3160 msgr2=0x7f1d4c0ac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c0b3160 0x7f1d4c0ac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 -- 172.21.15.3:0/1115231104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009a70 con 0x7f1d4c0717f0 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d51200700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c1611a0 0x7f1d4c0ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d4be00700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d4c0717f0 0x7f1d4c0ca6d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c4d0 tx=0x7f1d3c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:25.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d43e00700 1 -- 172.21.15.3:0/1115231104 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c005cd0 con 0x7f1d4c0717f0 2024-09-13T16:51:25.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d43e00700 1 -- 172.21.15.3:0/1115231104 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c005e30 con 0x7f1d4c0717f0 2024-09-13T16:51:25.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c0aad50 con 0x7f1d4c0717f0 2024-09-13T16:51:25.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d43e00700 1 -- 172.21.15.3:0/1115231104 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0072d0 con 0x7f1d4c0717f0 2024-09-13T16:51:25.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.772+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c0ab310 con 0x7f1d4c0717f0 2024-09-13T16:51:25.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.776+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c164d90 con 0x7f1d4c0717f0 2024-09-13T16:51:25.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.776+0000 7f1d43e00700 1 -- 172.21.15.3:0/1115231104 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d3c007430 con 0x7f1d4c0717f0 2024-09-13T16:51:25.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.776+0000 7f1d43e00700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1d2c066730 0x7f1d2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:25.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.780+0000 7f1d43e00700 1 -- 172.21.15.3:0/1115231104 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1d3c083920 con 0x7f1d4c0717f0 2024-09-13T16:51:25.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.780+0000 7f1d4b400700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1d2c066730 0x7f1d2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:25.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.780+0000 7f1d4b400700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1d2c066730 0x7f1d2c068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0a9760 tx=0x7f1d38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:25.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.784+0000 7f1d43e00700 1 -- 172.21.15.3:0/1115231104 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1d3c0508e0 con 0x7f1d4c0717f0 2024-09-13T16:51:25.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.968+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1d4c0cda70 con 0x7f1d4c0717f0 2024-09-13T16:51:25.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.968+0000 7f1d43e00700 1 -- 172.21.15.3:0/1115231104 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1d3c0170a0 con 0x7f1d4c0717f0 2024-09-13T16:51:25.974 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:25.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1d2c066730 msgr2=0x7f1d2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1d2c066730 0x7f1d2c068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0a9760 tx=0x7f1d38008040 comp rx=0 tx=0).stop 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d4c0717f0 msgr2=0x7f1d4c0ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d4c0717f0 0x7f1d4c0ca6d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c4d0 tx=0x7f1d3c00c890 comp rx=0 tx=0).stop 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 shutdown_connections 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1d2c066730 0x7f1d2c068bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4c1611a0 0x7f1d4c0ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c0b3160 0x7f1d4c0ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 --2- 172.21.15.3:0/1115231104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d4c0717f0 0x7f1d4c0ca6d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 >> 172.21.15.3:0/1115231104 conn(0x7f1d4c06b280 msgr2=0x7f1d4c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 shutdown_connections 2024-09-13T16:51:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:25.972+0000 7f1d53200700 1 -- 172.21.15.3:0/1115231104 wait complete. 2024-09-13T16:51:26.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:26.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.172+0000 7f8f2a400700 1 Processor -- start 2024-09-13T16:51:26.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.172+0000 7f8f2a400700 1 -- start start 2024-09-13T16:51:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.172+0000 7f8f2a400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f241443b0 0x7f8f240b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.172+0000 7f8f2a400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f24144790 0x7f8f2406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.172+0000 7f8f2a400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f240ce8e0 con 0x7f8f2406cb20 2024-09-13T16:51:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.172+0000 7f8f2a400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2406cf90 con 0x7f8f241443b0 2024-09-13T16:51:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.172+0000 7f8f2a400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240ccd30 con 0x7f8f24144790 2024-09-13T16:51:26.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f29a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8f2406cb20 0x7f8f2406a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51530/0 (socket says 172.21.15.3:51530) 2024-09-13T16:51:26.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f29a00700 1 -- 172.21.15.3:0/3235790770 learned_addr learned my addr 172.21.15.3:0/3235790770 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:26.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f29000700 1 --2- 172.21.15.3:0/3235790770 >> v2:172.21.15.157:3300/0 conn(0x7f8f241443b0 0x7f8f240b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f23e00700 1 --2- 172.21.15.3:0/3235790770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f24144790 0x7f8f2406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f23400700 1 -- 172.21.15.3:0/3235790770 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818046654 0 0) 0x7f8f240ce8e0 con 0x7f8f2406cb20 2024-09-13T16:51:26.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f23400700 1 -- 172.21.15.3:0/3235790770 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f0c0035d0 con 0x7f8f2406cb20 2024-09-13T16:51:26.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f29000700 1 -- 172.21.15.3:0/3235790770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f24144790 msgr2=0x7f8f2406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:26.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f29000700 1 --2- 172.21.15.3:0/3235790770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f24144790 0x7f8f2406c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f29000700 1 -- 172.21.15.3:0/3235790770 >> v1:172.21.15.3:6789/0 conn(0x7f8f2406cb20 legacy=0x7f8f2406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f29000700 1 -- 172.21.15.3:0/3235790770 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f240cd010 con 0x7f8f241443b0 2024-09-13T16:51:26.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f29000700 1 --2- 172.21.15.3:0/3235790770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f241443b0 0x7f8f240b39f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009850 tx=0x7f8f18009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d59a11e997d054 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f23400700 1 -- 172.21.15.3:0/3235790770 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18011830 con 0x7f8f241443b0 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f23400700 1 -- 172.21.15.3:0/3235790770 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f18011990 con 0x7f8f241443b0 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f23400700 1 -- 172.21.15.3:0/3235790770 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18011d40 con 0x7f8f241443b0 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f2a400700 1 -- 172.21.15.3:0/3235790770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f241443b0 msgr2=0x7f8f240b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f2a400700 1 --2- 172.21.15.3:0/3235790770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f241443b0 0x7f8f240b39f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009850 tx=0x7f8f18009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f2a400700 1 -- 172.21.15.3:0/3235790770 shutdown_connections 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f2a400700 1 --2- 172.21.15.3:0/3235790770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f24144790 0x7f8f2406c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f2a400700 1 --2- 172.21.15.3:0/3235790770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f241443b0 0x7f8f240b39f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f2a400700 1 -- 172.21.15.3:0/3235790770 >> 172.21.15.3:0/3235790770 conn(0x7f8f240b0cf0 msgr2=0x7f8f240b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f2a400700 1 -- 172.21.15.3:0/3235790770 shutdown_connections 2024-09-13T16:51:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.176+0000 7f8f2a400700 1 -- 172.21.15.3:0/3235790770 wait complete. 2024-09-13T16:51:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.180+0000 7f8f2a400700 1 Processor -- start 2024-09-13T16:51:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.180+0000 7f8f2a400700 1 -- start start 2024-09-13T16:51:26.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.180+0000 7f8f2a400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2406cb20 0x7f8f241e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.180+0000 7f8f2a400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f241443b0 0x7f8f241e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.180+0000 7f8f2a400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f240ae010 0x7f8f24070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.180+0000 7f8f2a400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2413bd10 con 0x7f8f2406cb20 2024-09-13T16:51:26.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.180+0000 7f8f2a400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f241e5410 con 0x7f8f240ae010 2024-09-13T16:51:26.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.180+0000 7f8f2a400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f241e3610 con 0x7f8f241443b0 2024-09-13T16:51:26.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2406cb20 0x7f8f241e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2406cb20 0x7f8f241e1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58486/0 (socket says 172.21.15.3:58486) 2024-09-13T16:51:26.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 -- 172.21.15.3:0/21273223 learned_addr learned my addr 172.21.15.3:0/21273223 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:26.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29a00700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f240ae010 0x7f8f24070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f23e00700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f241443b0 0x7f8f241e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 -- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f241443b0 msgr2=0x7f8f241e4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f241443b0 0x7f8f241e4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 -- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f240ae010 msgr2=0x7f8f24070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f240ae010 0x7f8f24070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 -- 172.21.15.3:0/21273223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f180094b0 con 0x7f8f2406cb20 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f23e00700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f241443b0 0x7f8f241e4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f29000700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2406cb20 0x7f8f241e1eb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f180090b0 tx=0x7f8f18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f20c00700 1 -- 172.21.15.3:0/21273223 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18004480 con 0x7f8f2406cb20 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f20c00700 1 -- 172.21.15.3:0/21273223 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1801f070 con 0x7f8f2406cb20 2024-09-13T16:51:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f241e2570 con 0x7f8f2406cb20 2024-09-13T16:51:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f20c00700 1 -- 172.21.15.3:0/21273223 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18003900 con 0x7f8f2406cb20 2024-09-13T16:51:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.184+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f241e2b30 con 0x7f8f2406cb20 2024-09-13T16:51:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.188+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f240af2c0 con 0x7f8f2406cb20 2024-09-13T16:51:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.188+0000 7f8f20c00700 1 -- 172.21.15.3:0/21273223 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8f18021030 con 0x7f8f2406cb20 2024-09-13T16:51:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.188+0000 7f8f20c00700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8f04066730 0x7f8f04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.188+0000 7f8f20c00700 1 -- 172.21.15.3:0/21273223 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8f18085890 con 0x7f8f2406cb20 2024-09-13T16:51:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.192+0000 7f8f23e00700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8f04066730 0x7f8f04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.192+0000 7f8f23e00700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8f04066730 0x7f8f04068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f241e3860 tx=0x7f8f10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.196+0000 7f8f20c00700 1 -- 172.21.15.3:0/21273223 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8f18052950 con 0x7f8f2406cb20 2024-09-13T16:51:26.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.404+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f240aefe0 con 0x7f8f2406cb20 2024-09-13T16:51:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f20c00700 1 -- 172.21.15.3:0/21273223 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f8f18006ce0 con 0x7f8f2406cb20 2024-09-13T16:51:26.412 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:26.413 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8f04066730 msgr2=0x7f8f04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8f04066730 0x7f8f04068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f241e3860 tx=0x7f8f10008040 comp rx=0 tx=0).stop 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2406cb20 msgr2=0x7f8f241e1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2406cb20 0x7f8f241e1eb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f180090b0 tx=0x7f8f18004770 comp rx=0 tx=0).stop 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 shutdown_connections 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8f04066730 0x7f8f04068bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f240ae010 0x7f8f24070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f241443b0 0x7f8f241e4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 --2- 172.21.15.3:0/21273223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2406cb20 0x7f8f241e1eb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 >> 172.21.15.3:0/21273223 conn(0x7f8f240b0cf0 msgr2=0x7f8f24143500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 shutdown_connections 2024-09-13T16:51:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.408+0000 7f8f2a400700 1 -- 172.21.15.3:0/21273223 wait complete. 2024-09-13T16:51:26.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:26.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.608+0000 7f9ac9c00700 1 Processor -- start 2024-09-13T16:51:26.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.608+0000 7f9ac9c00700 1 -- start start 2024-09-13T16:51:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.608+0000 7f9ac9c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ac415a970 0x7f9ac40d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.608+0000 7f9ac9c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40d0610 0x7f9ac415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.608+0000 7f9ac9c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ac40ce930 0x7f9ac415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.608+0000 7f9ac9c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac40b6100 con 0x7f9ac415a970 2024-09-13T16:51:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.608+0000 7f9ac9c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac40b5580 con 0x7f9ac40ce930 2024-09-13T16:51:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.608+0000 7f9ac9c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40b5860 con 0x7f9ac40d0610 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ac40ce930 0x7f9ac415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ac415a970 0x7f9ac40d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ac40ce930 0x7f9ac415eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:60244/0 (socket says 172.21.15.3:60244) 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ac415a970 0x7f9ac40d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58494/0 (socket says 172.21.15.3:58494) 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9200700 1 -- 172.21.15.3:0/2930250104 learned_addr learned my addr 172.21.15.3:0/2930250104 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3400700 1 --2- 172.21.15.3:0/2930250104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40d0610 0x7f9ac415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3e00700 1 -- 172.21.15.3:0/2930250104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40d0610 msgr2=0x7f9ac415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3e00700 1 --2- 172.21.15.3:0/2930250104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40d0610 0x7f9ac415e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3e00700 1 -- 172.21.15.3:0/2930250104 >> v2:172.21.15.157:3300/0 conn(0x7f9ac40ce930 msgr2=0x7f9ac415eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3e00700 1 --2- 172.21.15.3:0/2930250104 >> v2:172.21.15.157:3300/0 conn(0x7f9ac40ce930 0x7f9ac415eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3e00700 1 -- 172.21.15.3:0/2930250104 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac40b5b40 con 0x7f9ac415a970 2024-09-13T16:51:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9200700 1 --2- 172.21.15.3:0/2930250104 >> v2:172.21.15.157:3300/0 conn(0x7f9ac40ce930 0x7f9ac415eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac3e00700 1 --2- 172.21.15.3:0/2930250104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac40d00d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40b4710 tx=0x7f9ab4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb34aaf590b8df46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac2a00700 1 -- 172.21.15.3:0/2930250104 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4011830 con 0x7f9ac415a970 2024-09-13T16:51:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac2a00700 1 -- 172.21.15.3:0/2930250104 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab4011990 con 0x7f9ac415a970 2024-09-13T16:51:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac2a00700 1 -- 172.21.15.3:0/2930250104 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4011d20 con 0x7f9ac415a970 2024-09-13T16:51:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 -- 172.21.15.3:0/2930250104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 msgr2=0x7f9ac40d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/2930250104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac40d00d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40b4710 tx=0x7f9ab4009b30 comp rx=0 tx=0).stop 2024-09-13T16:51:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 -- 172.21.15.3:0/2930250104 shutdown_connections 2024-09-13T16:51:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/2930250104 >> v2:172.21.15.157:3300/0 conn(0x7f9ac40ce930 0x7f9ac415eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/2930250104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40d0610 0x7f9ac415e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/2930250104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac40d00d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 -- 172.21.15.3:0/2930250104 >> 172.21.15.3:0/2930250104 conn(0x7f9ac40723b0 msgr2=0x7f9ac406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 -- 172.21.15.3:0/2930250104 shutdown_connections 2024-09-13T16:51:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.612+0000 7f9ac9c00700 1 -- 172.21.15.3:0/2930250104 wait complete. 2024-09-13T16:51:26.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.616+0000 7f9ac9c00700 1 Processor -- start 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.616+0000 7f9ac9c00700 1 -- start start 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40ce930 0x7f9ac4152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac40d0610 0x7f9ac41f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41f4230 con 0x7f9ac415a970 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41f4510 con 0x7f9ac40d0610 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4152010 con 0x7f9ac40ce930 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac414faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58498/0 (socket says 172.21.15.3:58498) 2024-09-13T16:51:26.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9200700 1 -- 172.21.15.3:0/1836666467 learned_addr learned my addr 172.21.15.3:0/1836666467 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:26.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac3e00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40ce930 0x7f9ac4152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac3400700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac40d0610 0x7f9ac41f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac3e00700 1 -- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac40d0610 msgr2=0x7f9ac41f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac3e00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac40d0610 0x7f9ac41f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac3e00700 1 -- 172.21.15.3:0/1836666467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 msgr2=0x7f9ac414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac3e00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac414faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac3e00700 1 -- 172.21.15.3:0/1836666467 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc009b30 con 0x7f9ac40ce930 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9200700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac3e00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40ce930 0x7f9ac4152810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4016000 tx=0x7f9ab4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9abbe00700 1 -- 172.21.15.3:0/1836666467 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4004450 con 0x7f9ac40ce930 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9abbe00700 1 -- 172.21.15.3:0/1836666467 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab401e070 con 0x7f9ac40ce930 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9abbe00700 1 -- 172.21.15.3:0/1836666467 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4003900 con 0x7f9ac40ce930 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab40094b0 con 0x7f9ac40ce930 2024-09-13T16:51:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.620+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac4151d00 con 0x7f9ac40ce930 2024-09-13T16:51:26.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.624+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac41f4730 con 0x7f9ac40ce930 2024-09-13T16:51:26.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.624+0000 7f9abbe00700 1 -- 172.21.15.3:0/1836666467 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ab4003a60 con 0x7f9ac40ce930 2024-09-13T16:51:26.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.624+0000 7f9abbe00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9aa4066730 0x7f9aa4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:26.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.624+0000 7f9abbe00700 1 -- 172.21.15.3:0/1836666467 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9ab4085220 con 0x7f9ac40ce930 2024-09-13T16:51:26.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.624+0000 7f9ac3400700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9aa4066730 0x7f9aa4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:26.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.628+0000 7f9ac3400700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9aa4066730 0x7f9aa4068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ac414d210 tx=0x7f9ab0009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.632+0000 7f9abbe00700 1 -- 172.21.15.3:0/1836666467 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ab40521c0 con 0x7f9ac40ce930 2024-09-13T16:51:26.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.820+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ac4152530 con 0x7f9ac40ce930 2024-09-13T16:51:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.820+0000 7f9abbe00700 1 -- 172.21.15.3:0/1836666467 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ac4152530 con 0x7f9ac40ce930 2024-09-13T16:51:26.829 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.824+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9aa4066730 msgr2=0x7f9aa4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.824+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9aa4066730 0x7f9aa4068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ac414d210 tx=0x7f9ab0009240 comp rx=0 tx=0).stop 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.824+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40ce930 msgr2=0x7f9ac4152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.824+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40ce930 0x7f9ac4152810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4016000 tx=0x7f9ab4004770 comp rx=0 tx=0).stop 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.828+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 shutdown_connections 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.828+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9aa4066730 0x7f9aa4068bf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.828+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ac415a970 0x7f9ac414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.828+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac40d0610 0x7f9ac41f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.828+0000 7f9ac9c00700 1 --2- 172.21.15.3:0/1836666467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac40ce930 0x7f9ac4152810 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:26.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.828+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 >> 172.21.15.3:0/1836666467 conn(0x7f9ac40723b0 msgr2=0x7f9ac414bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:26.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.828+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 shutdown_connections 2024-09-13T16:51:26.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:26.828+0000 7f9ac9c00700 1 -- 172.21.15.3:0/1836666467 wait complete. 2024-09-13T16:51:26.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:27.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f6821800700 1 Processor -- start 2024-09-13T16:51:27.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f6821800700 1 -- start start 2024-09-13T16:51:27.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f6821800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f681c0b5570 0x7f681c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f6821800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c0b5a30 con 0x7f681c1444f0 2024-09-13T16:51:27.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f6821800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c0b5d10 con 0x7f681c0b0780 2024-09-13T16:51:27.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f6821800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c0cd5b0 con 0x7f681c0b5570 2024-09-13T16:51:27.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f681be00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f681c1444f0 0x7f681c0d0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51544/0 (socket says 172.21.15.3:51544) 2024-09-13T16:51:27.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f681be00700 1 -- 172.21.15.3:0/994105064 learned_addr learned my addr 172.21.15.3:0/994105064 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:27.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.036+0000 7f6813e00700 1 --2- 172.21.15.3:0/994105064 >> v2:172.21.15.157:3300/0 conn(0x7f681c0b5570 0x7f681c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712565584 0 0) 0x7f681c0b5a30 con 0x7f681c1444f0 2024-09-13T16:51:27.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68000035d0 con 0x7f681c1444f0 2024-09-13T16:51:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409814121 0 0) 0x7f68000035d0 con 0x7f681c1444f0 2024-09-13T16:51:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 >> v2:172.21.15.157:3300/0 conn(0x7f681c0b5570 msgr2=0x7f681c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 --2- 172.21.15.3:0/994105064 >> v2:172.21.15.157:3300/0 conn(0x7f681c0b5570 0x7f681c15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 >> v1:172.21.15.157:6789/0 conn(0x7f681c0b0780 legacy=0x7f681c0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0cd890 con 0x7f681c1444f0 2024-09-13T16:51:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f680c001670 con 0x7f681c1444f0 2024-09-13T16:51:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 >> v1:172.21.15.3:6789/0 conn(0x7f681c1444f0 legacy=0x7f681c0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f6800003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6800004420 0x7f68000048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68000050f0 0x7f680000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0cd5b0 con 0x7f6800003a20 2024-09-13T16:51:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680000c050 con 0x7f68000050f0 2024-09-13T16:51:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6800010620 con 0x7f6800004420 2024-09-13T16:51:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f6813e00700 1 --2- 172.21.15.3:0/994105064 >> v2:172.21.15.157:3300/0 conn(0x7f681c0b5570 0x7f681c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681be00700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f6800003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681be00700 1 -- 172.21.15.3:0/994105064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6800004420 msgr2=0x7f68000048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:27.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681be00700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6800004420 0x7f68000048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f6820e00700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68000050f0 0x7f680000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681be00700 1 -- 172.21.15.3:0/994105064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68000050f0 msgr2=0x7f680000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:27.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681be00700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68000050f0 0x7f680000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681be00700 1 -- 172.21.15.3:0/994105064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000035d0 con 0x7f6800003a20 2024-09-13T16:51:27.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f6820e00700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68000050f0 0x7f680000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f6820e00700 1 -- 172.21.15.3:0/994105064 reap_dead start 2024-09-13T16:51:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681be00700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f6800003e00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f680c00c2f0 tx=0x7f68000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49b64b19974a5d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c00ce30 con 0x7f6800003a20 2024-09-13T16:51:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c005980 con 0x7f6800003a20 2024-09-13T16:51:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.040+0000 7f681b400700 1 -- 172.21.15.3:0/994105064 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c005d30 con 0x7f6800003a20 2024-09-13T16:51:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 -- 172.21.15.3:0/994105064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 msgr2=0x7f6800003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f6800003e00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f680c00c2f0 tx=0x7f68000058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 -- 172.21.15.3:0/994105064 shutdown_connections 2024-09-13T16:51:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 --2- 172.21.15.3:0/994105064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f6800003e00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 -- 172.21.15.3:0/994105064 >> 172.21.15.3:0/994105064 conn(0x7f681c0721a0 msgr2=0x7f681c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 -- 172.21.15.3:0/994105064 shutdown_connections 2024-09-13T16:51:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 -- 172.21.15.3:0/994105064 wait complete. 2024-09-13T16:51:27.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 Processor -- start 2024-09-13T16:51:27.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.044+0000 7f6821800700 1 -- start start 2024-09-13T16:51:27.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6821800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f681c15b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6821800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f681c0c5b80 0x7f681c0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6821800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681c06b660 0x7f681c06bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6821800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c06d790 con 0x7f6800003a20 2024-09-13T16:51:27.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6821800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0c6540 con 0x7f681c06b660 2024-09-13T16:51:27.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6821800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c06c3b0 con 0x7f681c0c5b80 2024-09-13T16:51:27.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f681c15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f681c15b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58522/0 (socket says 172.21.15.3:58522) 2024-09-13T16:51:27.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 -- 172.21.15.3:0/862016089 learned_addr learned my addr 172.21.15.3:0/862016089 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:27.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6813e00700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f681c0c5b80 0x7f681c0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6820e00700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681c06b660 0x7f681c06bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 -- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f681c0c5b80 msgr2=0x7f681c0c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f681c0c5b80 0x7f681c0c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 -- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681c06b660 msgr2=0x7f681c06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681c06b660 0x7f681c06bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 -- 172.21.15.3:0/862016089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c06cdf0 con 0x7f6800003a20 2024-09-13T16:51:27.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f681be00700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f681c15b630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f680c000c00 tx=0x7f680c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6818c00700 1 -- 172.21.15.3:0/862016089 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c005d00 con 0x7f6800003a20 2024-09-13T16:51:27.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6818c00700 1 -- 172.21.15.3:0/862016089 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c005e60 con 0x7f6800003a20 2024-09-13T16:51:27.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6818c00700 1 -- 172.21.15.3:0/862016089 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c004ab0 con 0x7f6800003a20 2024-09-13T16:51:27.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c06d0d0 con 0x7f6800003a20 2024-09-13T16:51:27.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.048+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c15bb70 con 0x7f6800003a20 2024-09-13T16:51:27.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.052+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c0d0650 con 0x7f6800003a20 2024-09-13T16:51:27.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.052+0000 7f6818c00700 1 -- 172.21.15.3:0/862016089 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f680c004c10 con 0x7f6800003a20 2024-09-13T16:51:27.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.056+0000 7f6818c00700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f67f8066730 0x7f67f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.056+0000 7f6818c00700 1 -- 172.21.15.3:0/862016089 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f680c085650 con 0x7f6800003a20 2024-09-13T16:51:27.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.056+0000 7f6813e00700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f67f8066730 0x7f67f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.056+0000 7f6813e00700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f67f8066730 0x7f67f8068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f681c066960 tx=0x7f68040096e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.064+0000 7f6818c00700 1 -- 172.21.15.3:0/862016089 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f680c052710 con 0x7f6800003a20 2024-09-13T16:51:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.260+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f681c06a510 con 0x7f6800003a20 2024-09-13T16:51:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.260+0000 7f6818c00700 1 -- 172.21.15.3:0/862016089 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f680c007dc0 con 0x7f6800003a20 2024-09-13T16:51:27.267 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:27.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f67f8066730 msgr2=0x7f67f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f67f8066730 0x7f67f8068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f681c066960 tx=0x7f68040096e0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 msgr2=0x7f681c15b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f681c15b630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f680c000c00 tx=0x7f680c001a00 comp rx=0 tx=0).stop 2024-09-13T16:51:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 shutdown_connections 2024-09-13T16:51:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f67f8066730 0x7f67f8068bf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681c06b660 0x7f681c06bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f681c0c5b80 0x7f681c0c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 --2- 172.21.15.3:0/862016089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6800003a20 0x7f681c15b630 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.264+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 >> 172.21.15.3:0/862016089 conn(0x7f681c0721a0 msgr2=0x7f681c0cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.268+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 shutdown_connections 2024-09-13T16:51:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.268+0000 7f6821800700 1 -- 172.21.15.3:0/862016089 wait complete. 2024-09-13T16:51:27.294 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.476+0000 7effd0800700 1 Processor -- start 2024-09-13T16:51:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.476+0000 7effd0800700 1 -- start start 2024-09-13T16:51:27.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effd0800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effc81541b0 0x7effc815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effd0800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc806dac0 0x7effc815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effd0800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc8154590 con 0x7effc8070340 2024-09-13T16:51:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effd0800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc815e5d0 con 0x7effc81541b0 2024-09-13T16:51:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effd0800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc806e430 con 0x7effc806dac0 2024-09-13T16:51:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcd400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7effc8070340 0x7effc806d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51548/0 (socket says 172.21.15.3:51548) 2024-09-13T16:51:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcd400700 1 -- 172.21.15.3:0/2138326713 learned_addr learned my addr 172.21.15.3:0/2138326713 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcde00700 1 --2- 172.21.15.3:0/2138326713 >> v2:172.21.15.3:3300/0 conn(0x7effc81541b0 0x7effc815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effce800700 1 --2- 172.21.15.3:0/2138326713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc806dac0 0x7effc815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcca00700 1 -- 172.21.15.3:0/2138326713 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604057714 0 0) 0x7effc8154590 con 0x7effc8070340 2024-09-13T16:51:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcca00700 1 -- 172.21.15.3:0/2138326713 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effb00057e0 con 0x7effc8070340 2024-09-13T16:51:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcde00700 1 -- 172.21.15.3:0/2138326713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc806dac0 msgr2=0x7effc815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcde00700 1 --2- 172.21.15.3:0/2138326713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc806dac0 0x7effc815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcde00700 1 -- 172.21.15.3:0/2138326713 >> v1:172.21.15.3:6789/0 conn(0x7effc8070340 legacy=0x7effc806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcde00700 1 -- 172.21.15.3:0/2138326713 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc815dd30 con 0x7effc81541b0 2024-09-13T16:51:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effce800700 1 --2- 172.21.15.3:0/2138326713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc806dac0 0x7effc815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.480+0000 7effcde00700 1 --2- 172.21.15.3:0/2138326713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81541b0 0x7effc815a0b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7effbc009850 tx=0x7effbc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93c5848cca1593d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effcca00700 1 -- 172.21.15.3:0/2138326713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc011830 con 0x7effc81541b0 2024-09-13T16:51:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effcca00700 1 -- 172.21.15.3:0/2138326713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc011990 con 0x7effc81541b0 2024-09-13T16:51:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effcca00700 1 -- 172.21.15.3:0/2138326713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc011d00 con 0x7effc81541b0 2024-09-13T16:51:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effd0800700 1 -- 172.21.15.3:0/2138326713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81541b0 msgr2=0x7effc815a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effd0800700 1 --2- 172.21.15.3:0/2138326713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81541b0 0x7effc815a0b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7effbc009850 tx=0x7effbc009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effd0800700 1 -- 172.21.15.3:0/2138326713 shutdown_connections 2024-09-13T16:51:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effd0800700 1 --2- 172.21.15.3:0/2138326713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc806dac0 0x7effc815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effd0800700 1 --2- 172.21.15.3:0/2138326713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81541b0 0x7effc815a0b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effd0800700 1 -- 172.21.15.3:0/2138326713 >> 172.21.15.3:0/2138326713 conn(0x7effc806b280 msgr2=0x7effc806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effd0800700 1 -- 172.21.15.3:0/2138326713 shutdown_connections 2024-09-13T16:51:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.484+0000 7effd0800700 1 -- 172.21.15.3:0/2138326713 wait complete. 2024-09-13T16:51:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effd0800700 1 Processor -- start 2024-09-13T16:51:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effd0800700 1 -- start start 2024-09-13T16:51:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effd0800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc806dac0 0x7effc8155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effd0800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc8070340 0x7effc815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effd0800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc81541b0 0x7effc815b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effd0800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc8157cb0 con 0x7effc806dac0 2024-09-13T16:51:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effd0800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc8158270 con 0x7effc81541b0 2024-09-13T16:51:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effd0800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc8155d60 con 0x7effc8070340 2024-09-13T16:51:27.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effcde00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc806dac0 0x7effc8155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effcde00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc806dac0 0x7effc8155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58534/0 (socket says 172.21.15.3:58534) 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effcde00700 1 -- 172.21.15.3:0/812947393 learned_addr learned my addr 172.21.15.3:0/812947393 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effcd400700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc8070340 0x7effc815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.488+0000 7effce800700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc81541b0 0x7effc815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effcde00700 1 -- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc8070340 msgr2=0x7effc815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effcde00700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc8070340 0x7effc815a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effcde00700 1 -- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc81541b0 msgr2=0x7effc815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effcde00700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc81541b0 0x7effc815b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effcde00700 1 -- 172.21.15.3:0/812947393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4009a70 con 0x7effc806dac0 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effce800700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc81541b0 0x7effc815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effcd400700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc8070340 0x7effc815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effcde00700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc806dac0 0x7effc8155820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7effc80669f0 tx=0x7effbc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effc2000700 1 -- 172.21.15.3:0/812947393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc004550 con 0x7effc806dac0 2024-09-13T16:51:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effc2000700 1 -- 172.21.15.3:0/812947393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc01e070 con 0x7effc806dac0 2024-09-13T16:51:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effc2000700 1 -- 172.21.15.3:0/812947393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc003900 con 0x7effc806dac0 2024-09-13T16:51:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc0094b0 con 0x7effc806dac0 2024-09-13T16:51:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc815bff0 con 0x7effc806dac0 2024-09-13T16:51:27.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.492+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc815d6a0 con 0x7effc806dac0 2024-09-13T16:51:27.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.496+0000 7effc2000700 1 -- 172.21.15.3:0/812947393 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effbc003a60 con 0x7effc806dac0 2024-09-13T16:51:27.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.496+0000 7effc2000700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7effac066760 0x7effac068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.496+0000 7effc2000700 1 -- 172.21.15.3:0/812947393 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7effbc085400 con 0x7effc806dac0 2024-09-13T16:51:27.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.496+0000 7effcd400700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7effac066760 0x7effac068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.500+0000 7effcd400700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7effac066760 0x7effac068c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effc815cbf0 tx=0x7effb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.504+0000 7effc2000700 1 -- 172.21.15.3:0/812947393 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7effbc0523a0 con 0x7effc806dac0 2024-09-13T16:51:27.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.680+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7effc8070120 con 0x7effc806dac0 2024-09-13T16:51:27.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.680+0000 7effc2000700 1 -- 172.21.15.3:0/812947393 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7effbc0559c0 con 0x7effc806dac0 2024-09-13T16:51:27.687 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:27.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7effac066760 msgr2=0x7effac068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7effac066760 0x7effac068c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effc815cbf0 tx=0x7effb8009040 comp rx=0 tx=0).stop 2024-09-13T16:51:27.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc806dac0 msgr2=0x7effc8155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc806dac0 0x7effc8155820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7effc80669f0 tx=0x7effbc004770 comp rx=0 tx=0).stop 2024-09-13T16:51:27.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 shutdown_connections 2024-09-13T16:51:27.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7effac066760 0x7effac068c20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc81541b0 0x7effc815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc8070340 0x7effc815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 --2- 172.21.15.3:0/812947393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc806dac0 0x7effc8155820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 >> 172.21.15.3:0/812947393 conn(0x7effc806b280 msgr2=0x7effc806c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:27.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 shutdown_connections 2024-09-13T16:51:27.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.684+0000 7effd0800700 1 -- 172.21.15.3:0/812947393 wait complete. 2024-09-13T16:51:27.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.892+0000 7fe97ee00700 1 Processor -- start 2024-09-13T16:51:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.892+0000 7fe97ee00700 1 -- start start 2024-09-13T16:51:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97ee00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9780b4510 0x7fe97815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97ee00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9780b48f0 0x7fe9780cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97ee00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97815e1d0 con 0x7fe9780cd5b0 2024-09-13T16:51:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97ee00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9780cc670 con 0x7fe9780b4510 2024-09-13T16:51:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97ee00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9780b56a0 con 0x7fe9780b48f0 2024-09-13T16:51:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97da00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9780b4510 0x7fe97815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97e400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe9780cd5b0 0x7fe9780cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51556/0 (socket says 172.21.15.3:51556) 2024-09-13T16:51:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97e400700 1 -- 172.21.15.3:0/1482535792 learned_addr learned my addr 172.21.15.3:0/1482535792 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97d000700 1 --2- 172.21.15.3:0/1482535792 >> v2:172.21.15.157:3300/0 conn(0x7fe9780b48f0 0x7fe9780cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97da00700 1 -- 172.21.15.3:0/1482535792 >> v2:172.21.15.157:3300/0 conn(0x7fe9780b48f0 msgr2=0x7fe9780cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97da00700 1 --2- 172.21.15.3:0/1482535792 >> v2:172.21.15.157:3300/0 conn(0x7fe9780b48f0 0x7fe9780cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97da00700 1 -- 172.21.15.3:0/1482535792 >> v1:172.21.15.3:6789/0 conn(0x7fe9780cd5b0 legacy=0x7fe9780cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97da00700 1 -- 172.21.15.3:0/1482535792 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780b5980 con 0x7fe9780b4510 2024-09-13T16:51:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe977e00700 1 -- 172.21.15.3:0/1482535792 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72638417 0 0) 0x7fe97815e1d0 con 0x7fe9780cd5b0 2024-09-13T16:51:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe97da00700 1 --2- 172.21.15.3:0/1482535792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b4510 0x7fe97815dc90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe978066e70 tx=0x7fe96c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a106269eb190d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe977e00700 1 -- 172.21.15.3:0/1482535792 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c011830 con 0x7fe9780b4510 2024-09-13T16:51:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe977e00700 1 -- 172.21.15.3:0/1482535792 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c011990 con 0x7fe9780b4510 2024-09-13T16:51:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.896+0000 7fe977e00700 1 -- 172.21.15.3:0/1482535792 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c011d20 con 0x7fe9780b4510 2024-09-13T16:51:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 -- 172.21.15.3:0/1482535792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b4510 msgr2=0x7fe97815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 --2- 172.21.15.3:0/1482535792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b4510 0x7fe97815dc90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe978066e70 tx=0x7fe96c009b30 comp rx=0 tx=0).stop 2024-09-13T16:51:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 -- 172.21.15.3:0/1482535792 shutdown_connections 2024-09-13T16:51:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 --2- 172.21.15.3:0/1482535792 >> v2:172.21.15.157:3300/0 conn(0x7fe9780b48f0 0x7fe9780cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 --2- 172.21.15.3:0/1482535792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b4510 0x7fe97815dc90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 -- 172.21.15.3:0/1482535792 >> 172.21.15.3:0/1482535792 conn(0x7fe978071da0 msgr2=0x7fe9780b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 -- 172.21.15.3:0/1482535792 shutdown_connections 2024-09-13T16:51:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 -- 172.21.15.3:0/1482535792 wait complete. 2024-09-13T16:51:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 Processor -- start 2024-09-13T16:51:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.900+0000 7fe97ee00700 1 -- start start 2024-09-13T16:51:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97ee00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9780b4510 0x7fe978153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97ee00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b48f0 0x7fe9780ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97ee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9780cd5b0 0x7fe9780abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97ee00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9781f6300 con 0x7fe9780b48f0 2024-09-13T16:51:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97ee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe978159660 con 0x7fe9780cd5b0 2024-09-13T16:51:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97ee00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9781f3470 con 0x7fe9780b4510 2024-09-13T16:51:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b48f0 0x7fe9780ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b48f0 0x7fe9780ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58560/0 (socket says 172.21.15.3:58560) 2024-09-13T16:51:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 -- 172.21.15.3:0/3651268969 learned_addr learned my addr 172.21.15.3:0/3651268969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97e400700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9780cd5b0 0x7fe9780abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97da00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9780b4510 0x7fe978153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 -- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9780b4510 msgr2=0x7fe978153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9780b4510 0x7fe978153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 -- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9780cd5b0 msgr2=0x7fe9780abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9780cd5b0 0x7fe9780abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 -- 172.21.15.3:0/3651268969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0094b0 con 0x7fe9780b48f0 2024-09-13T16:51:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97d000700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b48f0 0x7fe9780ab730 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e9d0 tx=0x7fe96400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe975600700 1 -- 172.21.15.3:0/3651268969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964009720 con 0x7fe9780b48f0 2024-09-13T16:51:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe975600700 1 -- 172.21.15.3:0/3651268969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96400ee70 con 0x7fe9780b48f0 2024-09-13T16:51:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9781f3190 con 0x7fe9780b48f0 2024-09-13T16:51:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe975600700 1 -- 172.21.15.3:0/3651268969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400ca50 con 0x7fe9780b48f0 2024-09-13T16:51:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.904+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9781f5f20 con 0x7fe9780b48f0 2024-09-13T16:51:27.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.908+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97806edc0 con 0x7fe9780b48f0 2024-09-13T16:51:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.908+0000 7fe975600700 1 -- 172.21.15.3:0/3651268969 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe96400cbb0 con 0x7fe9780b48f0 2024-09-13T16:51:27.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.912+0000 7fe975600700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe958066730 0x7fe958068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:27.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.912+0000 7fe975600700 1 -- 172.21.15.3:0/3651268969 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe9640844d0 con 0x7fe9780b48f0 2024-09-13T16:51:27.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.912+0000 7fe97da00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe958066730 0x7fe958068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.912+0000 7fe97da00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe958066730 0x7fe958068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe96c000c00 tx=0x7fe96c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:27.916+0000 7fe975600700 1 -- 172.21.15.3:0/3651268969 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe964051470 con 0x7fe9780b48f0 2024-09-13T16:51:28.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.100+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe9780a85f0 con 0x7fe9780b48f0 2024-09-13T16:51:28.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.100+0000 7fe975600700 1 -- 172.21.15.3:0/3651268969 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe96400ce60 con 0x7fe9780b48f0 2024-09-13T16:51:28.106 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe958066730 msgr2=0x7fe958068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe958066730 0x7fe958068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe96c000c00 tx=0x7fe96c008030 comp rx=0 tx=0).stop 2024-09-13T16:51:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b48f0 msgr2=0x7fe9780ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b48f0 0x7fe9780ab730 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e9d0 tx=0x7fe96400ece0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 shutdown_connections 2024-09-13T16:51:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe958066730 0x7fe958068bf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9780cd5b0 0x7fe9780abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9780b48f0 0x7fe9780ab730 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 --2- 172.21.15.3:0/3651268969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9780b4510 0x7fe978153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 >> 172.21.15.3:0/3651268969 conn(0x7fe978071da0 msgr2=0x7fe97815bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 shutdown_connections 2024-09-13T16:51:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.104+0000 7fe97ee00700 1 -- 172.21.15.3:0/3651268969 wait complete. 2024-09-13T16:51:28.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57f200700 1 Processor -- start 2024-09-13T16:51:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57f200700 1 -- start start 2024-09-13T16:51:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57f200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57815cbc0 0x7fd5780adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57f200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd57815cfa0 0x7fd5780ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57f200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57806a390 con 0x7fd578070080 2024-09-13T16:51:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57f200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57806a670 con 0x7fd57815cfa0 2024-09-13T16:51:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57f200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57806a950 con 0x7fd57815cbc0 2024-09-13T16:51:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57e800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd578070080 0x7fd5780ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51570/0 (socket says 172.21.15.3:51570) 2024-09-13T16:51:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57e800700 1 -- 172.21.15.3:0/2540155694 learned_addr learned my addr 172.21.15.3:0/2540155694 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57d400700 1 --2- 172.21.15.3:0/2540155694 >> v2:172.21.15.3:3300/0 conn(0x7fd57815cfa0 0x7fd5780ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57de00700 1 --2- 172.21.15.3:0/2540155694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57815cbc0 0x7fd5780adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.304+0000 7fd57ca00700 1 -- 172.21.15.3:0/2540155694 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605222668 0 0) 0x7fd57806a390 con 0x7fd578070080 2024-09-13T16:51:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57ca00700 1 -- 172.21.15.3:0/2540155694 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5600035d0 con 0x7fd578070080 2024-09-13T16:51:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57de00700 1 -- 172.21.15.3:0/2540155694 >> v2:172.21.15.3:3300/0 conn(0x7fd57815cfa0 msgr2=0x7fd5780ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57de00700 1 --2- 172.21.15.3:0/2540155694 >> v2:172.21.15.3:3300/0 conn(0x7fd57815cfa0 0x7fd5780ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57de00700 1 -- 172.21.15.3:0/2540155694 >> v1:172.21.15.3:6789/0 conn(0x7fd578070080 legacy=0x7fd5780ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57de00700 1 -- 172.21.15.3:0/2540155694 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57806ac30 con 0x7fd57815cbc0 2024-09-13T16:51:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57de00700 1 --2- 172.21.15.3:0/2540155694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57815cbc0 0x7fd5780adad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009850 tx=0x7fd56c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62534c0c0bffcc92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:28.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57ca00700 1 -- 172.21.15.3:0/2540155694 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c011830 con 0x7fd57815cbc0 2024-09-13T16:51:28.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57ca00700 1 -- 172.21.15.3:0/2540155694 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c011990 con 0x7fd57815cbc0 2024-09-13T16:51:28.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57ca00700 1 -- 172.21.15.3:0/2540155694 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c011d40 con 0x7fd57815cbc0 2024-09-13T16:51:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57f200700 1 -- 172.21.15.3:0/2540155694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57815cbc0 msgr2=0x7fd5780adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57f200700 1 --2- 172.21.15.3:0/2540155694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57815cbc0 0x7fd5780adad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009850 tx=0x7fd56c009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57f200700 1 -- 172.21.15.3:0/2540155694 shutdown_connections 2024-09-13T16:51:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57f200700 1 --2- 172.21.15.3:0/2540155694 >> v2:172.21.15.3:3300/0 conn(0x7fd57815cfa0 0x7fd5780ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57f200700 1 --2- 172.21.15.3:0/2540155694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57815cbc0 0x7fd5780adad0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57f200700 1 -- 172.21.15.3:0/2540155694 >> 172.21.15.3:0/2540155694 conn(0x7fd5780b0cf0 msgr2=0x7fd5780b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57f200700 1 -- 172.21.15.3:0/2540155694 shutdown_connections 2024-09-13T16:51:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.308+0000 7fd57f200700 1 -- 172.21.15.3:0/2540155694 wait complete. 2024-09-13T16:51:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57f200700 1 Processor -- start 2024-09-13T16:51:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57f200700 1 -- start start 2024-09-13T16:51:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57f200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd578070080 0x7fd578150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57f200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd57815cbc0 0x7fd57814df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57815cfa0 0x7fd57815d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57f200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57814e450 con 0x7fd57815cbc0 2024-09-13T16:51:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57815e480 con 0x7fd57815cfa0 2024-09-13T16:51:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57f200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57815dbe0 con 0x7fd578070080 2024-09-13T16:51:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd57815cbc0 0x7fd57814df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd57815cbc0 0x7fd57814df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58588/0 (socket says 172.21.15.3:58588) 2024-09-13T16:51:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57d400700 1 -- 172.21.15.3:0/1057781831 learned_addr learned my addr 172.21.15.3:0/1057781831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57de00700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd578070080 0x7fd578150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57e800700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57815cfa0 0x7fd57815d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57d400700 1 -- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd578070080 msgr2=0x7fd578150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57d400700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd578070080 0x7fd578150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57d400700 1 -- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57815cfa0 msgr2=0x7fd57815d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57d400700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57815cfa0 0x7fd57815d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57d400700 1 -- 172.21.15.3:0/1057781831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c0094b0 con 0x7fd57815cbc0 2024-09-13T16:51:28.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.312+0000 7fd57e800700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57815cfa0 0x7fd57815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:28.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.316+0000 7fd57d400700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd57815cbc0 0x7fd57814df10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd56800e840 tx=0x7fd56800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:28.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.316+0000 7fd572000700 1 -- 172.21.15.3:0/1057781831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568013680 con 0x7fd57815cbc0 2024-09-13T16:51:28.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.316+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57815dec0 con 0x7fd57815cbc0 2024-09-13T16:51:28.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.316+0000 7fd572000700 1 -- 172.21.15.3:0/1057781831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5680137e0 con 0x7fd57815cbc0 2024-09-13T16:51:28.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.316+0000 7fd572000700 1 -- 172.21.15.3:0/1057781831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568013b90 con 0x7fd57815cbc0 2024-09-13T16:51:28.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.316+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd578151130 con 0x7fd57815cbc0 2024-09-13T16:51:28.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.316+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd578160f50 con 0x7fd57815cbc0 2024-09-13T16:51:28.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.320+0000 7fd572000700 1 -- 172.21.15.3:0/1057781831 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd568013cf0 con 0x7fd57815cbc0 2024-09-13T16:51:28.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.320+0000 7fd572000700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd55c066730 0x7fd55c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.320+0000 7fd572000700 1 -- 172.21.15.3:0/1057781831 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd5680844d0 con 0x7fd57815cbc0 2024-09-13T16:51:28.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.320+0000 7fd57de00700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd55c066730 0x7fd55c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.324+0000 7fd57de00700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd55c066730 0x7fd55c068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0090b0 tx=0x7fd56c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.328+0000 7fd572000700 1 -- 172.21.15.3:0/1057781831 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd5680513f0 con 0x7fd57815cbc0 2024-09-13T16:51:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.524+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5780ccb60 con 0x7fd57815cbc0 2024-09-13T16:51:28.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd572000700 1 -- 172.21.15.3:0/1057781831 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7fd568054a10 con 0x7fd57815cbc0 2024-09-13T16:51:28.532 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:28.532 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd55c066730 msgr2=0x7fd55c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd55c066730 0x7fd55c068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0090b0 tx=0x7fd56c008030 comp rx=0 tx=0).stop 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd57815cbc0 msgr2=0x7fd57814df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd57815cbc0 0x7fd57814df10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd56800e840 tx=0x7fd56800eb50 comp rx=0 tx=0).stop 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 shutdown_connections 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd55c066730 0x7fd55c068bf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57815cfa0 0x7fd57815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd57815cbc0 0x7fd57814df10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 --2- 172.21.15.3:0/1057781831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd578070080 0x7fd578150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 >> 172.21.15.3:0/1057781831 conn(0x7fd5780b0cf0 msgr2=0x7fd578160840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 shutdown_connections 2024-09-13T16:51:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.528+0000 7fd57f200700 1 -- 172.21.15.3:0/1057781831 wait complete. 2024-09-13T16:51:28.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:51:28.550 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:51:28.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:28.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.772+0000 7f5fc8000700 1 Processor -- start 2024-09-13T16:51:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.772+0000 7f5fc8000700 1 -- start start 2024-09-13T16:51:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.776+0000 7f5fc8000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5fc015e7c0 0x7f5fc015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.776+0000 7f5fc8000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 0x7f5fc00b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.776+0000 7f5fc8000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc015b7c0 con 0x7f5fc0155250 2024-09-13T16:51:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.776+0000 7f5fc8000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00b5c20 con 0x7f5fc015e7c0 2024-09-13T16:51:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.776+0000 7f5fc8000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00ce1e0 con 0x7f5fc00b5250 2024-09-13T16:51:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.776+0000 7f5fc5600700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5fc0155250 0x7f5fc015bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51586/0 (socket says 172.21.15.3:51586) 2024-09-13T16:51:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.776+0000 7f5fc5600700 1 -- 172.21.15.3:0/1503324548 learned_addr learned my addr 172.21.15.3:0/1503324548 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc4c00700 1 --2- 172.21.15.3:0/1503324548 >> v2:172.21.15.157:3300/0 conn(0x7f5fc015e7c0 0x7f5fc015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc6000700 1 --2- 172.21.15.3:0/1503324548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 0x7f5fc00b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fbfe00700 1 -- 172.21.15.3:0/1503324548 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655671366 0 0) 0x7f5fc015b7c0 con 0x7f5fc0155250 2024-09-13T16:51:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fbfe00700 1 -- 172.21.15.3:0/1503324548 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fa80035d0 con 0x7f5fc0155250 2024-09-13T16:51:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc4c00700 1 -- 172.21.15.3:0/1503324548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 msgr2=0x7f5fc00b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc4c00700 1 --2- 172.21.15.3:0/1503324548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 0x7f5fc00b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc4c00700 1 -- 172.21.15.3:0/1503324548 >> v1:172.21.15.3:6789/0 conn(0x7f5fc0155250 legacy=0x7f5fc015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc4c00700 1 -- 172.21.15.3:0/1503324548 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc015b1d0 con 0x7f5fc015e7c0 2024-09-13T16:51:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc4c00700 1 --2- 172.21.15.3:0/1503324548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 0x7f5fc015ec20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009850 tx=0x7f5fac009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d056764677a1909 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fbfe00700 1 -- 172.21.15.3:0/1503324548 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac011830 con 0x7f5fc015e7c0 2024-09-13T16:51:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fbfe00700 1 -- 172.21.15.3:0/1503324548 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac011990 con 0x7f5fc015e7c0 2024-09-13T16:51:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fbfe00700 1 -- 172.21.15.3:0/1503324548 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac011d40 con 0x7f5fc015e7c0 2024-09-13T16:51:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc8000700 1 -- 172.21.15.3:0/1503324548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 msgr2=0x7f5fc015ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc8000700 1 --2- 172.21.15.3:0/1503324548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 0x7f5fc015ec20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009850 tx=0x7f5fac009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc8000700 1 -- 172.21.15.3:0/1503324548 shutdown_connections 2024-09-13T16:51:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc8000700 1 --2- 172.21.15.3:0/1503324548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 0x7f5fc00b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc8000700 1 --2- 172.21.15.3:0/1503324548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 0x7f5fc015ec20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.780+0000 7f5fc8000700 1 -- 172.21.15.3:0/1503324548 >> 172.21.15.3:0/1503324548 conn(0x7f5fc0071f10 msgr2=0x7f5fc006b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 -- 172.21.15.3:0/1503324548 shutdown_connections 2024-09-13T16:51:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 -- 172.21.15.3:0/1503324548 wait complete. 2024-09-13T16:51:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 Processor -- start 2024-09-13T16:51:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 -- start start 2024-09-13T16:51:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 0x7f5fc0157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc0155250 0x7f5fc0149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 0x7f5fc00c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00ca720 con 0x7f5fc0155250 2024-09-13T16:51:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0158370 con 0x7f5fc015e7c0 2024-09-13T16:51:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.784+0000 7f5fc8000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0149d50 con 0x7f5fc00b5250 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc0155250 0x7f5fc0149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc0155250 0x7f5fc0149810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58594/0 (socket says 172.21.15.3:58594) 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 -- 172.21.15.3:0/2455828264 learned_addr learned my addr 172.21.15.3:0/2455828264 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc5600700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 0x7f5fc0157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc6000700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 0x7f5fc00c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 -- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 msgr2=0x7f5fc0157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 0x7f5fc0157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 -- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 msgr2=0x7f5fc00c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 0x7f5fc00c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 -- 172.21.15.3:0/2455828264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0094b0 con 0x7f5fc0155250 2024-09-13T16:51:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc4c00700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc0155250 0x7f5fc0149810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5fac014040 tx=0x7f5fac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fbd600700 1 -- 172.21.15.3:0/2455828264 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac004510 con 0x7f5fc0155250 2024-09-13T16:51:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fbd600700 1 -- 172.21.15.3:0/2455828264 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac01e070 con 0x7f5fc0155250 2024-09-13T16:51:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc00c9800 con 0x7f5fc0155250 2024-09-13T16:51:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fbd600700 1 -- 172.21.15.3:0/2455828264 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac003900 con 0x7f5fc0155250 2024-09-13T16:51:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.788+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc00c9d80 con 0x7f5fc0155250 2024-09-13T16:51:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.792+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc00cf8d0 con 0x7f5fc0155250 2024-09-13T16:51:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.792+0000 7f5fbd600700 1 -- 172.21.15.3:0/2455828264 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5fac003a60 con 0x7f5fc0155250 2024-09-13T16:51:28.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.796+0000 7f5fbd600700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5fa0066730 0x7f5fa0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:28.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.796+0000 7f5fbd600700 1 -- 172.21.15.3:0/2455828264 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5fac020070 con 0x7f5fc0155250 2024-09-13T16:51:28.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.796+0000 7f5fc5600700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5fa0066730 0x7f5fa0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.796+0000 7f5fc5600700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5fa0066730 0x7f5fa0068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4007fd0 tx=0x7f5fb4007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:28.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.800+0000 7f5fbd600700 1 -- 172.21.15.3:0/2455828264 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5fac052820 con 0x7f5fc0155250 2024-09-13T16:51:28.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.980+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fc00b59b0 con 0x7f5fc0155250 2024-09-13T16:51:28.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.980+0000 7f5fbd600700 1 -- 172.21.15.3:0/2455828264 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fac006ce0 con 0x7f5fc0155250 2024-09-13T16:51:28.985 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.980+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5fa0066730 msgr2=0x7f5fa0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.980+0000 7f5fc8000700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5fa0066730 0x7f5fa0068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4007fd0 tx=0x7f5fb4007de0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc0155250 msgr2=0x7f5fc0149810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc0155250 0x7f5fc0149810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5fac014040 tx=0x7f5fac004770 comp rx=0 tx=0).stop 2024-09-13T16:51:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 shutdown_connections 2024-09-13T16:51:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5fa0066730 0x7f5fa0068bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc015e7c0 0x7f5fc00c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc0155250 0x7f5fc0149810 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 --2- 172.21.15.3:0/2455828264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fc00b5250 0x7f5fc0157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:28.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 >> 172.21.15.3:0/2455828264 conn(0x7f5fc0071f10 msgr2=0x7f5fc015dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:28.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 shutdown_connections 2024-09-13T16:51:28.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:28.984+0000 7f5fc8000700 1 -- 172.21.15.3:0/2455828264 wait complete. 2024-09-13T16:51:29.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:29.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.192+0000 7f3f96600700 1 Processor -- start 2024-09-13T16:51:29.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.192+0000 7f3f96600700 1 -- start start 2024-09-13T16:51:29.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f96600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f90158740 0x7f3f900ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f96600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f900b1f30 con 0x7f3f9015b1c0 2024-09-13T16:51:29.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f96600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f9015b710 con 0x7f3f900cc830 2024-09-13T16:51:29.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f96600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f900b3960 con 0x7f3f90158740 2024-09-13T16:51:29.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8f400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f90158740 0x7f3f900ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8f400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f90158740 0x7f3f900ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58602/0 (socket says 172.21.15.3:58602) 2024-09-13T16:51:29.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8f400700 1 -- 172.21.15.3:0/4197647742 learned_addr learned my addr 172.21.15.3:0/4197647742 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:29.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8ea00700 1 --1- 172.21.15.3:0/4197647742 >> v1:172.21.15.3:6789/0 conn(0x7f3f9015b1c0 0x7f3f900b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51590/0 (socket says 172.21.15.3:51590) 2024-09-13T16:51:29.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8f400700 1 -- 172.21.15.3:0/4197647742 >> v1:172.21.15.157:6789/0 conn(0x7f3f900cc830 legacy=0x7f3f900ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8f400700 1 -- 172.21.15.3:0/4197647742 >> v1:172.21.15.3:6789/0 conn(0x7f3f9015b1c0 legacy=0x7f3f900b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8f400700 1 -- 172.21.15.3:0/4197647742 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9015b9f0 con 0x7f3f90158740 2024-09-13T16:51:29.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8e000700 1 -- 172.21.15.3:0/4197647742 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988176869 0 0) 0x7f3f900b1f30 con 0x7f3f9015b1c0 2024-09-13T16:51:29.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8f400700 1 --2- 172.21.15.3:0/4197647742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f90158740 0x7f3f900ce6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009910 tx=0x7f3f80009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c0f5383b464a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8e000700 1 -- 172.21.15.3:0/4197647742 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000ba60 con 0x7f3f90158740 2024-09-13T16:51:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8e000700 1 -- 172.21.15.3:0/4197647742 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8000bbc0 con 0x7f3f90158740 2024-09-13T16:51:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.196+0000 7f3f8e000700 1 -- 172.21.15.3:0/4197647742 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80009e60 con 0x7f3f90158740 2024-09-13T16:51:29.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 -- 172.21.15.3:0/4197647742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f90158740 msgr2=0x7f3f900ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 --2- 172.21.15.3:0/4197647742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f90158740 0x7f3f900ce6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009910 tx=0x7f3f80009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:29.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 -- 172.21.15.3:0/4197647742 shutdown_connections 2024-09-13T16:51:29.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 --2- 172.21.15.3:0/4197647742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f90158740 0x7f3f900ce6e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 -- 172.21.15.3:0/4197647742 >> 172.21.15.3:0/4197647742 conn(0x7f3f9006b280 msgr2=0x7f3f9006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 -- 172.21.15.3:0/4197647742 shutdown_connections 2024-09-13T16:51:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 -- 172.21.15.3:0/4197647742 wait complete. 2024-09-13T16:51:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 Processor -- start 2024-09-13T16:51:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.200+0000 7f3f96600700 1 -- start start 2024-09-13T16:51:29.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f900cc830 0x7f3f900ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f90158740 0x7f3f901f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9015b1c0 0x7f3f901f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f901f3730 con 0x7f3f9015b1c0 2024-09-13T16:51:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900aea60 con 0x7f3f900cc830 2024-09-13T16:51:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f901f5a50 con 0x7f3f90158740 2024-09-13T16:51:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9015b1c0 0x7f3f901f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9015b1c0 0x7f3f901f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58612/0 (socket says 172.21.15.3:58612) 2024-09-13T16:51:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 -- 172.21.15.3:0/619222069 learned_addr learned my addr 172.21.15.3:0/619222069 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8f400700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f900cc830 0x7f3f900ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8ea00700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f90158740 0x7f3f901f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 -- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f90158740 msgr2=0x7f3f901f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f90158740 0x7f3f901f31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 -- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f900cc830 msgr2=0x7f3f900ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f900cc830 0x7f3f900ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 -- 172.21.15.3:0/619222069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80009570 con 0x7f3f9015b1c0 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8ea00700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f90158740 0x7f3f901f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f8fe00700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9015b1c0 0x7f3f901f5510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800cc50 tx=0x7f3f8800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f87400700 1 -- 172.21.15.3:0/619222069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88003c20 con 0x7f3f9015b1c0 2024-09-13T16:51:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f87400700 1 -- 172.21.15.3:0/619222069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f88003d80 con 0x7f3f9015b1c0 2024-09-13T16:51:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f900acf50 con 0x7f3f9015b1c0 2024-09-13T16:51:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f87400700 1 -- 172.21.15.3:0/619222069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800ab60 con 0x7f3f9015b1c0 2024-09-13T16:51:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f900ad510 con 0x7f3f9015b1c0 2024-09-13T16:51:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.204+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f900ce6e0 con 0x7f3f9015b1c0 2024-09-13T16:51:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.208+0000 7f3f87400700 1 -- 172.21.15.3:0/619222069 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f8800acc0 con 0x7f3f9015b1c0 2024-09-13T16:51:29.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.212+0000 7f3f87400700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f70066730 0x7f3f70068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.212+0000 7f3f87400700 1 -- 172.21.15.3:0/619222069 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3f88084600 con 0x7f3f9015b1c0 2024-09-13T16:51:29.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.212+0000 7f3f8f400700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f70066730 0x7f3f70068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.212+0000 7f3f8f400700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f70066730 0x7f3f70068bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f80000c00 tx=0x7f3f80008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:29.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.216+0000 7f3f87400700 1 -- 172.21.15.3:0/619222069 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f88051510 con 0x7f3f9015b1c0 2024-09-13T16:51:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.404+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f900aec20 con 0x7f3f9015b1c0 2024-09-13T16:51:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.404+0000 7f3f87400700 1 -- 172.21.15.3:0/619222069 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f88054b30 con 0x7f3f9015b1c0 2024-09-13T16:51:29.410 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f70066730 msgr2=0x7f3f70068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f70066730 0x7f3f70068bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f80000c00 tx=0x7f3f80008030 comp rx=0 tx=0).stop 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9015b1c0 msgr2=0x7f3f901f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9015b1c0 0x7f3f901f5510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800cc50 tx=0x7f3f8800cf60 comp rx=0 tx=0).stop 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 shutdown_connections 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f70066730 0x7f3f70068bf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9015b1c0 0x7f3f901f5510 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f90158740 0x7f3f901f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 --2- 172.21.15.3:0/619222069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f900cc830 0x7f3f900ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 >> 172.21.15.3:0/619222069 conn(0x7f3f9006b280 msgr2=0x7f3f900b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:29.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 shutdown_connections 2024-09-13T16:51:29.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.408+0000 7f3f96600700 1 -- 172.21.15.3:0/619222069 wait complete. 2024-09-13T16:51:29.438 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.608+0000 7f7282600700 1 Processor -- start 2024-09-13T16:51:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.608+0000 7f7282600700 1 -- start start 2024-09-13T16:51:29.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.608+0000 7f7282600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c15e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.608+0000 7f7282600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f727c06de80 0x7f727c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f7282600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727c15ae00 con 0x7f727c06a390 2024-09-13T16:51:29.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f7282600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727c06e2c0 con 0x7f727c06de80 2024-09-13T16:51:29.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f7282600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c0b5330 con 0x7f727c15aa20 2024-09-13T16:51:29.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727be00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f727c06a390 0x7f727c0b2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51600/0 (socket says 172.21.15.3:51600) 2024-09-13T16:51:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727be00700 1 -- 172.21.15.3:0/9815236 learned_addr learned my addr 172.21.15.3:0/9815236 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727aa00700 1 --2- 172.21.15.3:0/9815236 >> v2:172.21.15.3:3300/0 conn(0x7f727c06de80 0x7f727c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727b400700 1 --2- 172.21.15.3:0/9815236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c15e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727a000700 1 -- 172.21.15.3:0/9815236 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749024197 0 0) 0x7f727c15ae00 con 0x7f727c06a390 2024-09-13T16:51:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727a000700 1 -- 172.21.15.3:0/9815236 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72600035d0 con 0x7f727c06a390 2024-09-13T16:51:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727aa00700 1 -- 172.21.15.3:0/9815236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 msgr2=0x7f727c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727aa00700 1 --2- 172.21.15.3:0/9815236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c15e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727aa00700 1 -- 172.21.15.3:0/9815236 >> v1:172.21.15.3:6789/0 conn(0x7f727c06a390 legacy=0x7f727c0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727aa00700 1 -- 172.21.15.3:0/9815236 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c0b5610 con 0x7f727c06de80 2024-09-13T16:51:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727b400700 1 --2- 172.21.15.3:0/9815236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c15e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727aa00700 1 --2- 172.21.15.3:0/9815236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06de80 0x7f727c0b4b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7268009850 tx=0x7f7268009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74efed5d93d335b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727a000700 1 -- 172.21.15.3:0/9815236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268011830 con 0x7f727c06de80 2024-09-13T16:51:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727a000700 1 -- 172.21.15.3:0/9815236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7268011990 con 0x7f727c06de80 2024-09-13T16:51:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.612+0000 7f727a000700 1 -- 172.21.15.3:0/9815236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268011d00 con 0x7f727c06de80 2024-09-13T16:51:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 -- 172.21.15.3:0/9815236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06de80 msgr2=0x7f727c0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 --2- 172.21.15.3:0/9815236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06de80 0x7f727c0b4b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7268009850 tx=0x7f7268009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 -- 172.21.15.3:0/9815236 shutdown_connections 2024-09-13T16:51:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 --2- 172.21.15.3:0/9815236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06de80 0x7f727c0b4b00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 --2- 172.21.15.3:0/9815236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c15e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 -- 172.21.15.3:0/9815236 >> 172.21.15.3:0/9815236 conn(0x7f727c071f10 msgr2=0x7f727c06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 -- 172.21.15.3:0/9815236 shutdown_connections 2024-09-13T16:51:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 -- 172.21.15.3:0/9815236 wait complete. 2024-09-13T16:51:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.616+0000 7f7282600700 1 Processor -- start 2024-09-13T16:51:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7282600700 1 -- start start 2024-09-13T16:51:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7282600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06a390 0x7f727c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7282600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f727c06de80 0x7f727c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7282600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7282600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c157cc0 con 0x7f727c06a390 2024-09-13T16:51:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7282600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c157fa0 con 0x7f727c06de80 2024-09-13T16:51:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7282600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c158280 con 0x7f727c15aa20 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06a390 0x7f727c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06a390 0x7f727c153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58628/0 (socket says 172.21.15.3:58628) 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 -- 172.21.15.3:0/3875338197 learned_addr learned my addr 172.21.15.3:0/3875338197 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727be00700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727aa00700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f727c06de80 0x7f727c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 -- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 msgr2=0x7f727c0b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c0b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 -- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f727c06de80 msgr2=0x7f727c158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f727c06de80 0x7f727c158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 -- 172.21.15.3:0/3875338197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0094b0 con 0x7f727c06a390 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727aa00700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f727c06de80 0x7f727c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f727b400700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06a390 0x7f727c153ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f726c00ebe0 tx=0x7f726c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7273400700 1 -- 172.21.15.3:0/3875338197 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c00c890 con 0x7f727c06a390 2024-09-13T16:51:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7273400700 1 -- 172.21.15.3:0/3875338197 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c00c9f0 con 0x7f727c06a390 2024-09-13T16:51:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.620+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72680094b0 con 0x7f727c06a390 2024-09-13T16:51:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.624+0000 7f7273400700 1 -- 172.21.15.3:0/3875338197 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c00cd80 con 0x7f727c06a390 2024-09-13T16:51:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.624+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c154b90 con 0x7f727c06a390 2024-09-13T16:51:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.624+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727c0b4b00 con 0x7f727c06a390 2024-09-13T16:51:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.624+0000 7f7273400700 1 -- 172.21.15.3:0/3875338197 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f726c013470 con 0x7f727c06a390 2024-09-13T16:51:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.628+0000 7f7273400700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f725c066730 0x7f725c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.628+0000 7f7273400700 1 -- 172.21.15.3:0/3875338197 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f726c0845b0 con 0x7f727c06a390 2024-09-13T16:51:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.628+0000 7f727aa00700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f725c066730 0x7f725c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.628+0000 7f727aa00700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f725c066730 0x7f725c068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7268009850 tx=0x7f726801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:29.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.632+0000 7f7273400700 1 -- 172.21.15.3:0/3875338197 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f726c0515f0 con 0x7f727c06a390 2024-09-13T16:51:29.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.816+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f727c06e480 con 0x7f727c06a390 2024-09-13T16:51:29.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.816+0000 7f7273400700 1 -- 172.21.15.3:0/3875338197 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f726c054c10 con 0x7f727c06a390 2024-09-13T16:51:29.821 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:29.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f725c066730 msgr2=0x7f725c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f725c066730 0x7f725c068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7268009850 tx=0x7f726801c040 comp rx=0 tx=0).stop 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06a390 msgr2=0x7f727c153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06a390 0x7f727c153ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f726c00ebe0 tx=0x7f726c00eef0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 shutdown_connections 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f725c066730 0x7f725c068bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727c15aa20 0x7f727c0b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f727c06de80 0x7f727c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 --2- 172.21.15.3:0/3875338197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727c06a390 0x7f727c153ad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 >> 172.21.15.3:0/3875338197 conn(0x7f727c071f10 msgr2=0x7f727c06c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 shutdown_connections 2024-09-13T16:51:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:29.820+0000 7f7282600700 1 -- 172.21.15.3:0/3875338197 wait complete. 2024-09-13T16:51:29.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.032+0000 7f7ca2200700 1 Processor -- start 2024-09-13T16:51:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.032+0000 7f7ca2200700 1 -- start start 2024-09-13T16:51:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca2200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c15b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca2200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9c0b41e0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca2200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9c06a390 con 0x7f7c9c1442c0 2024-09-13T16:51:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca2200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c06b1f0 con 0x7f7c9c0cdc30 2024-09-13T16:51:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7c9be00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7c9c0cd7c0 0x7f7c9c15b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51612/0 (socket says 172.21.15.3:51612) 2024-09-13T16:51:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7c9be00700 1 -- 172.21.15.3:0/959780658 learned_addr learned my addr 172.21.15.3:0/959780658 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:30.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca1800700 1 --2- 172.21.15.3:0/959780658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7c9b400700 1 -- 172.21.15.3:0/959780658 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744501522 0 0) 0x7f7c9c0b41e0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7c9b400700 1 -- 172.21.15.3:0/959780658 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c840035d0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca1800700 1 -- 172.21.15.3:0/959780658 >> v1:172.21.15.157:6789/0 conn(0x7f7c9c1442c0 legacy=0x7f7c9c06d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7c9b400700 1 -- 172.21.15.3:0/959780658 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491590494 0 0) 0x7f7c9c06a390 con 0x7f7c9c1442c0 2024-09-13T16:51:30.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca1800700 1 -- 172.21.15.3:0/959780658 >> v1:172.21.15.3:6789/0 conn(0x7f7c9c0cd7c0 legacy=0x7f7c9c15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca1800700 1 -- 172.21.15.3:0/959780658 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c06a5e0 con 0x7f7c9c0cdc30 2024-09-13T16:51:30.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7ca1800700 1 --2- 172.21.15.3:0/959780658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c15b820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009dc0 tx=0x7f7c9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cfed8adea8c5dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7c9b400700 1 -- 172.21.15.3:0/959780658 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400e070 con 0x7f7c9c0cdc30 2024-09-13T16:51:30.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7c9b400700 1 -- 172.21.15.3:0/959780658 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9400bb20 con 0x7f7c9c0cdc30 2024-09-13T16:51:30.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.036+0000 7f7c9b400700 1 -- 172.21.15.3:0/959780658 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400be90 con 0x7f7c9c0cdc30 2024-09-13T16:51:30.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 -- 172.21.15.3:0/959780658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 msgr2=0x7f7c9c15b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 --2- 172.21.15.3:0/959780658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c15b820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009dc0 tx=0x7f7c9400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:30.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 -- 172.21.15.3:0/959780658 shutdown_connections 2024-09-13T16:51:30.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 --2- 172.21.15.3:0/959780658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c15b820 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 -- 172.21.15.3:0/959780658 >> 172.21.15.3:0/959780658 conn(0x7f7c9c0af620 msgr2=0x7f7c9c0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:30.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 -- 172.21.15.3:0/959780658 shutdown_connections 2024-09-13T16:51:30.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 -- 172.21.15.3:0/959780658 wait complete. 2024-09-13T16:51:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 Processor -- start 2024-09-13T16:51:30.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.040+0000 7f7ca2200700 1 -- start start 2024-09-13T16:51:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca2200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c0cd7c0 0x7f7c9c13d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca2200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c0a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca2200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c1442c0 0x7f7c9c15a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca2200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c13cba0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca2200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c13bf90 con 0x7f7c9c1442c0 2024-09-13T16:51:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca2200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c13c270 con 0x7f7c9c0cdc30 2024-09-13T16:51:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c0cd7c0 0x7f7c9c13d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c0cd7c0 0x7f7c9c13d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58644/0 (socket says 172.21.15.3:58644) 2024-09-13T16:51:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 -- 172.21.15.3:0/1746849862 learned_addr learned my addr 172.21.15.3:0/1746849862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca1800700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c1442c0 0x7f7c9c15a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7c9be00700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c0a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 -- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 msgr2=0x7f7c9c0a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c0a8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 -- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c1442c0 msgr2=0x7f7c9c15a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c1442c0 0x7f7c9c15a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 -- 172.21.15.3:0/1746849862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c94009a70 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7c9be00700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c0a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca1800700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c1442c0 0x7f7c9c15a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca0e00700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c0cd7c0 0x7f7c9c13d030 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c520 tx=0x7f7c9000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7c98c00700 1 -- 172.21.15.3:0/1746849862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c900054e0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7c98c00700 1 -- 172.21.15.3:0/1746849862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c900036a0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c13c550 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7c98c00700 1 -- 172.21.15.3:0/1746849862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90003a50 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.044+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9c13fcb0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.048+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9c1446a0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.048+0000 7f7c98c00700 1 -- 172.21.15.3:0/1746849862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7c90003bb0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.052+0000 7f7c98c00700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7c7c066730 0x7f7c7c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.052+0000 7f7c98c00700 1 -- 172.21.15.3:0/1746849862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7c90083a30 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.052+0000 7f7c9be00700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7c7c066730 0x7f7c7c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.052+0000 7f7c9be00700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7c7c066730 0x7f7c7c068bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c139df0 tx=0x7f7c88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.056+0000 7f7c98c00700 1 -- 172.21.15.3:0/1746849862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7c900509d0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.236+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7c9c15e300 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.236+0000 7f7c98c00700 1 -- 172.21.15.3:0/1746849862 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7c900170a0 con 0x7f7c9c0cd7c0 2024-09-13T16:51:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:30.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7c7c066730 msgr2=0x7f7c7c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7c7c066730 0x7f7c7c068bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c139df0 tx=0x7f7c88009040 comp rx=0 tx=0).stop 2024-09-13T16:51:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c0cd7c0 msgr2=0x7f7c9c13d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c0cd7c0 0x7f7c9c13d030 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c520 tx=0x7f7c9000c8e0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 shutdown_connections 2024-09-13T16:51:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7c7c066730 0x7f7c7c068bf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c1442c0 0x7f7c9c15a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c9c0cdc30 0x7f7c9c0a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 --2- 172.21.15.3:0/1746849862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c0cd7c0 0x7f7c9c13d030 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 >> 172.21.15.3:0/1746849862 conn(0x7f7c9c0af620 msgr2=0x7f7c9c06b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:30.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 shutdown_connections 2024-09-13T16:51:30.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.240+0000 7f7ca2200700 1 -- 172.21.15.3:0/1746849862 wait complete. 2024-09-13T16:51:30.290 INFO:tasks.cephfs_test_runner:Starting test: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) The maximum number of openfiles omap objects keys are now equal to 2024-09-13T16:51:30.291 INFO:tasks.cephfs_test_runner:test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) 2024-09-13T16:51:30.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.460+0000 7f2af7200700 1 Processor -- start 2024-09-13T16:51:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.460+0000 7f2af7200700 1 -- start start 2024-09-13T16:51:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af7200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af80ae1e0 0x7f2af80cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af7200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af806a390 con 0x7f2af815cfa0 2024-09-13T16:51:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af7200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af806a670 con 0x7f2af815cbc0 2024-09-13T16:51:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af7200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af806a950 con 0x7f2af80ae1e0 2024-09-13T16:51:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af5400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2af815cfa0 0x7f2af80ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51616/0 (socket says 172.21.15.3:51616) 2024-09-13T16:51:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af5400700 1 -- 172.21.15.3:0/852601563 learned_addr learned my addr 172.21.15.3:0/852601563 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af6800700 1 --2- 172.21.15.3:0/852601563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af80ae1e0 0x7f2af80cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af4a00700 1 -- 172.21.15.3:0/852601563 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920191403 0 0) 0x7f2af806a390 con 0x7f2af815cfa0 2024-09-13T16:51:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af4a00700 1 -- 172.21.15.3:0/852601563 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2adc0035d0 con 0x7f2af815cfa0 2024-09-13T16:51:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.464+0000 7f2af4a00700 1 -- 172.21.15.3:0/852601563 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912821267 0 0) 0x7f2af806a670 con 0x7f2af815cbc0 2024-09-13T16:51:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af4a00700 1 -- 172.21.15.3:0/852601563 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af806a390 con 0x7f2af815cbc0 2024-09-13T16:51:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af6800700 1 -- 172.21.15.3:0/852601563 >> v1:172.21.15.157:6789/0 conn(0x7f2af815cbc0 legacy=0x7f2af80adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af6800700 1 -- 172.21.15.3:0/852601563 >> v1:172.21.15.3:6789/0 conn(0x7f2af815cfa0 legacy=0x7f2af80ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af6800700 1 -- 172.21.15.3:0/852601563 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af806ac30 con 0x7f2af80ae1e0 2024-09-13T16:51:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af6800700 1 --2- 172.21.15.3:0/852601563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af80ae1e0 0x7f2af80cd270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009dc0 tx=0x7f2af000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f19d51bfd852ab79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af4a00700 1 -- 172.21.15.3:0/852601563 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000e070 con 0x7f2af80ae1e0 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af4a00700 1 -- 172.21.15.3:0/852601563 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000bb20 con 0x7f2af80ae1e0 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af4a00700 1 -- 172.21.15.3:0/852601563 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0012430 con 0x7f2af80ae1e0 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af7200700 1 -- 172.21.15.3:0/852601563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af80ae1e0 msgr2=0x7f2af80cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af7200700 1 --2- 172.21.15.3:0/852601563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af80ae1e0 0x7f2af80cd270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009dc0 tx=0x7f2af000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af7200700 1 -- 172.21.15.3:0/852601563 shutdown_connections 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af7200700 1 --2- 172.21.15.3:0/852601563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af80ae1e0 0x7f2af80cd270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af7200700 1 -- 172.21.15.3:0/852601563 >> 172.21.15.3:0/852601563 conn(0x7f2af80b0cf0 msgr2=0x7f2af80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af7200700 1 -- 172.21.15.3:0/852601563 shutdown_connections 2024-09-13T16:51:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.468+0000 7f2af7200700 1 -- 172.21.15.3:0/852601563 wait complete. 2024-09-13T16:51:30.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af7200700 1 Processor -- start 2024-09-13T16:51:30.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af7200700 1 -- start start 2024-09-13T16:51:30.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af7200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af80ae1e0 0x7f2af81f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af7200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af815cbc0 0x7f2af81577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af815cfa0 0x7f2af8157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af7200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af815e090 con 0x7f2af80ae1e0 2024-09-13T16:51:30.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af815d450 con 0x7f2af815cfa0 2024-09-13T16:51:30.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af7200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af815d6f0 con 0x7f2af815cbc0 2024-09-13T16:51:30.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af80ae1e0 0x7f2af81f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af80ae1e0 0x7f2af81f2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58654/0 (socket says 172.21.15.3:58654) 2024-09-13T16:51:30.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 -- 172.21.15.3:0/4110618756 learned_addr learned my addr 172.21.15.3:0/4110618756 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:30.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5400700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af815cbc0 0x7f2af81577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af6800700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af815cfa0 0x7f2af8157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 -- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af815cbc0 msgr2=0x7f2af81577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af815cbc0 0x7f2af81577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 -- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af815cfa0 msgr2=0x7f2af8157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af815cfa0 0x7f2af8157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 -- 172.21.15.3:0/4110618756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0009a70 con 0x7f2af80ae1e0 2024-09-13T16:51:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.472+0000 7f2af5e00700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af80ae1e0 0x7f2af81f2190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c730 tx=0x7f2ae800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.476+0000 7f2aee000700 1 -- 172.21.15.3:0/4110618756 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80052d0 con 0x7f2af80ae1e0 2024-09-13T16:51:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.476+0000 7f2aee000700 1 -- 172.21.15.3:0/4110618756 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8005450 con 0x7f2af80ae1e0 2024-09-13T16:51:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.476+0000 7f2aee000700 1 -- 172.21.15.3:0/4110618756 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80038e0 con 0x7f2af80ae1e0 2024-09-13T16:51:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.476+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af815d9d0 con 0x7f2af80ae1e0 2024-09-13T16:51:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.476+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af81f2880 con 0x7f2af80ae1e0 2024-09-13T16:51:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.476+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af81f26d0 con 0x7f2af80ae1e0 2024-09-13T16:51:30.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.480+0000 7f2aee000700 1 -- 172.21.15.3:0/4110618756 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ae8003a40 con 0x7f2af80ae1e0 2024-09-13T16:51:30.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.480+0000 7f2aee000700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad8066730 0x7f2ad8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.480+0000 7f2aee000700 1 -- 172.21.15.3:0/4110618756 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2ae8083970 con 0x7f2af80ae1e0 2024-09-13T16:51:30.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.480+0000 7f2af5400700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad8066730 0x7f2ad8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.484+0000 7f2af5400700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad8066730 0x7f2ad8068bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2af81567c0 tx=0x7f2ae4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.488+0000 7f2aee000700 1 -- 172.21.15.3:0/4110618756 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2ae8050910 con 0x7f2af80ae1e0 2024-09-13T16:51:30.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.688+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2af80cdfd0 con 0x7f2af80ae1e0 2024-09-13T16:51:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.688+0000 7f2aee000700 1 -- 172.21.15.3:0/4110618756 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f2ae8081e60 con 0x7f2af80ae1e0 2024-09-13T16:51:30.696 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:30.696 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad8066730 msgr2=0x7f2ad8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad8066730 0x7f2ad8068bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2af81567c0 tx=0x7f2ae4009040 comp rx=0 tx=0).stop 2024-09-13T16:51:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af80ae1e0 msgr2=0x7f2af81f2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af80ae1e0 0x7f2af81f2190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c730 tx=0x7f2ae800ca40 comp rx=0 tx=0).stop 2024-09-13T16:51:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 shutdown_connections 2024-09-13T16:51:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad8066730 0x7f2ad8068bf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af815cfa0 0x7f2af8157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af815cbc0 0x7f2af81577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 --2- 172.21.15.3:0/4110618756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af80ae1e0 0x7f2af81f2190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 >> 172.21.15.3:0/4110618756 conn(0x7f2af80b0cf0 msgr2=0x7f2af80b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:30.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 shutdown_connections 2024-09-13T16:51:30.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.696+0000 7f2af7200700 1 -- 172.21.15.3:0/4110618756 wait complete. 2024-09-13T16:51:30.728 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:30.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.900+0000 7fb7e4400700 1 Processor -- start 2024-09-13T16:51:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.900+0000 7fb7e4400700 1 -- start start 2024-09-13T16:51:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e4400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc1610e0 0x7fb7dc06f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e4400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7dc161bd0 0x7fb7dc1632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e4400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc0702f0 con 0x7fb7dc06feb0 2024-09-13T16:51:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e4400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7dc162000 con 0x7fb7dc161bd0 2024-09-13T16:51:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e4400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc162250 con 0x7fb7dc1610e0 2024-09-13T16:51:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e2400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7dc161bd0 0x7fb7dc1632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e1a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc1610e0 0x7fb7dc06f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e1a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc1610e0 0x7fb7dc06f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:34738/0 (socket says 172.21.15.3:34738) 2024-09-13T16:51:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e1a00700 1 -- 172.21.15.3:0/3483083278 learned_addr learned my addr 172.21.15.3:0/3483083278 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e2400700 1 -- 172.21.15.3:0/3483083278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc1610e0 msgr2=0x7fb7dc06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e2400700 1 --2- 172.21.15.3:0/3483083278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc1610e0 0x7fb7dc06f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e2400700 1 -- 172.21.15.3:0/3483083278 >> v1:172.21.15.157:6789/0 conn(0x7fb7dc06feb0 legacy=0x7fb7dc1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e2400700 1 -- 172.21.15.3:0/3483083278 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc162530 con 0x7fb7dc161bd0 2024-09-13T16:51:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e1a00700 1 --2- 172.21.15.3:0/3483083278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc1610e0 0x7fb7dc06f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7e2400700 1 --2- 172.21.15.3:0/3483083278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc161bd0 0x7fb7dc1632f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009d90 tx=0x7fb7d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d5f43b41d3c532 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7dbe00700 1 -- 172.21.15.3:0/3483083278 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d400e070 con 0x7fb7dc161bd0 2024-09-13T16:51:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7dbe00700 1 -- 172.21.15.3:0/3483083278 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d400bb20 con 0x7fb7dc161bd0 2024-09-13T16:51:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.904+0000 7fb7dbe00700 1 -- 172.21.15.3:0/3483083278 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4012430 con 0x7fb7dc161bd0 2024-09-13T16:51:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 -- 172.21.15.3:0/3483083278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc161bd0 msgr2=0x7fb7dc1632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 --2- 172.21.15.3:0/3483083278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc161bd0 0x7fb7dc1632f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009d90 tx=0x7fb7d400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 -- 172.21.15.3:0/3483083278 shutdown_connections 2024-09-13T16:51:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 --2- 172.21.15.3:0/3483083278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc161bd0 0x7fb7dc1632f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 --2- 172.21.15.3:0/3483083278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc1610e0 0x7fb7dc06f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 -- 172.21.15.3:0/3483083278 >> 172.21.15.3:0/3483083278 conn(0x7fb7dc0b2b00 msgr2=0x7fb7dc0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 -- 172.21.15.3:0/3483083278 shutdown_connections 2024-09-13T16:51:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 -- 172.21.15.3:0/3483083278 wait complete. 2024-09-13T16:51:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 Processor -- start 2024-09-13T16:51:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.908+0000 7fb7e4400700 1 -- start start 2024-09-13T16:51:30.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e4400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc06feb0 0x7fb7dc154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e4400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc1610e0 0x7fb7dc1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e4400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7dc161bd0 0x7fb7dc151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e4400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc154a40 con 0x7fb7dc1610e0 2024-09-13T16:51:30.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e4400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc154d20 con 0x7fb7dc161bd0 2024-09-13T16:51:30.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e4400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc155000 con 0x7fb7dc06feb0 2024-09-13T16:51:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc1610e0 0x7fb7dc1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc1610e0 0x7fb7dc1535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58672/0 (socket says 172.21.15.3:58672) 2024-09-13T16:51:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 -- 172.21.15.3:0/892574385 learned_addr learned my addr 172.21.15.3:0/892574385 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e2400700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7dc161bd0 0x7fb7dc151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1a00700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc06feb0 0x7fb7dc154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 -- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc06feb0 msgr2=0x7fb7dc154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc06feb0 0x7fb7dc154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 -- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7dc161bd0 msgr2=0x7fb7dc151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7dc161bd0 0x7fb7dc151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 -- 172.21.15.3:0/892574385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d0009570 con 0x7fb7dc1610e0 2024-09-13T16:51:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e1000700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc1610e0 0x7fb7dc1535d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c500 tx=0x7fb7c800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7d9600700 1 -- 172.21.15.3:0/892574385 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80052d0 con 0x7fb7dc1610e0 2024-09-13T16:51:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7d9600700 1 -- 172.21.15.3:0/892574385 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c8005430 con 0x7fb7dc1610e0 2024-09-13T16:51:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7d9600700 1 -- 172.21.15.3:0/892574385 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80055a0 con 0x7fb7dc1610e0 2024-09-13T16:51:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d4009a70 con 0x7fb7dc1610e0 2024-09-13T16:51:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.912+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc1555c0 con 0x7fb7dc1610e0 2024-09-13T16:51:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.916+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7dc06f970 con 0x7fb7dc1610e0 2024-09-13T16:51:30.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.916+0000 7fb7d9600700 1 -- 172.21.15.3:0/892574385 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb7c80036a0 con 0x7fb7dc1610e0 2024-09-13T16:51:30.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.920+0000 7fb7d9600700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7bc066730 0x7fb7bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:30.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.920+0000 7fb7d9600700 1 -- 172.21.15.3:0/892574385 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb7c80836f0 con 0x7fb7dc1610e0 2024-09-13T16:51:30.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.920+0000 7fb7e1a00700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7bc066730 0x7fb7bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:30.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.920+0000 7fb7e1a00700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7bc066730 0x7fb7bc068bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006d50 tx=0x7fb7d00092e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:30.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:30.924+0000 7fb7d9600700 1 -- 172.21.15.3:0/892574385 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb7c8050690 con 0x7fb7dc1610e0 2024-09-13T16:51:31.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.108+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7dc155780 con 0x7fb7dc1610e0 2024-09-13T16:51:31.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.108+0000 7fb7d9600700 1 -- 172.21.15.3:0/892574385 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7c8053cb0 con 0x7fb7dc1610e0 2024-09-13T16:51:31.116 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:31.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7bc066730 msgr2=0x7fb7bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7bc066730 0x7fb7bc068bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006d50 tx=0x7fb7d00092e0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc1610e0 msgr2=0x7fb7dc1535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc1610e0 0x7fb7dc1535d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c500 tx=0x7fb7c800c810 comp rx=0 tx=0).stop 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 shutdown_connections 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7bc066730 0x7fb7bc068bf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7dc161bd0 0x7fb7dc151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7dc1610e0 0x7fb7dc1535d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 --2- 172.21.15.3:0/892574385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7dc06feb0 0x7fb7dc154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 >> 172.21.15.3:0/892574385 conn(0x7fb7dc0b2b00 msgr2=0x7fb7dc0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 shutdown_connections 2024-09-13T16:51:31.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.112+0000 7fb7e4400700 1 -- 172.21.15.3:0/892574385 wait complete. 2024-09-13T16:51:31.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:31.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.312+0000 7fc3fcc00700 1 Processor -- start 2024-09-13T16:51:31.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.312+0000 7fc3fcc00700 1 -- start start 2024-09-13T16:51:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3fcc00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc3f806dac0 0x7fc3f815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3fcc00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f8070340 0x7fc3f815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3fcc00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f8154590 con 0x7fc3f81541b0 2024-09-13T16:51:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3fcc00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f815e5d0 con 0x7fc3f806dac0 2024-09-13T16:51:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3fcc00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f806e430 con 0x7fc3f8070340 2024-09-13T16:51:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f5400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc3f806dac0 0x7fc3f815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f5e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc3f81541b0 0x7fc3f806d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:46808/0 (socket says 172.21.15.3:46808) 2024-09-13T16:51:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f5e00700 1 -- 172.21.15.3:0/3180920094 learned_addr learned my addr 172.21.15.3:0/3180920094 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f6800700 1 --2- 172.21.15.3:0/3180920094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f8070340 0x7fc3f815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f5400700 1 -- 172.21.15.3:0/3180920094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f8070340 msgr2=0x7fc3f815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:31.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f5400700 1 --2- 172.21.15.3:0/3180920094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f8070340 0x7fc3f815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f5400700 1 -- 172.21.15.3:0/3180920094 >> v1:172.21.15.157:6789/0 conn(0x7fc3f81541b0 legacy=0x7fc3f806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f5400700 1 -- 172.21.15.3:0/3180920094 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f815dd30 con 0x7fc3f806dac0 2024-09-13T16:51:31.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f4a00700 1 -- 172.21.15.3:0/3180920094 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232903804 0 0) 0x7fc3f8154590 con 0x7fc3f81541b0 2024-09-13T16:51:31.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.316+0000 7fc3f5400700 1 --2- 172.21.15.3:0/3180920094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f806dac0 0x7fc3f815a0b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009850 tx=0x7fc3e4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5858e01d04f019 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3f4a00700 1 -- 172.21.15.3:0/3180920094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4011830 con 0x7fc3f806dac0 2024-09-13T16:51:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3f4a00700 1 -- 172.21.15.3:0/3180920094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e4011990 con 0x7fc3f806dac0 2024-09-13T16:51:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3f4a00700 1 -- 172.21.15.3:0/3180920094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4011d20 con 0x7fc3f806dac0 2024-09-13T16:51:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3180920094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f806dac0 msgr2=0x7fc3f815a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3180920094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f806dac0 0x7fc3f815a0b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009850 tx=0x7fc3e4009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3180920094 shutdown_connections 2024-09-13T16:51:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3180920094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f8070340 0x7fc3f815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3180920094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f806dac0 0x7fc3f815a0b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3180920094 >> 172.21.15.3:0/3180920094 conn(0x7fc3f806b280 msgr2=0x7fc3f806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3180920094 shutdown_connections 2024-09-13T16:51:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.320+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3180920094 wait complete. 2024-09-13T16:51:31.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3fcc00700 1 Processor -- start 2024-09-13T16:51:31.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3fcc00700 1 -- start start 2024-09-13T16:51:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3fcc00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f806dac0 0x7fc3f81f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3fcc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f8070340 0x7fc3f8148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3fcc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f81541b0 0x7fc3f80ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3fcc00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f815c190 con 0x7fc3f81541b0 2024-09-13T16:51:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3fcc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81f6cf0 con 0x7fc3f8070340 2024-09-13T16:51:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3fcc00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f815b580 con 0x7fc3f806dac0 2024-09-13T16:51:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3f6800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f81541b0 0x7fc3f80ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3f6800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f81541b0 0x7fc3f80ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58700/0 (socket says 172.21.15.3:58700) 2024-09-13T16:51:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3f6800700 1 -- 172.21.15.3:0/3036198771 learned_addr learned my addr 172.21.15.3:0/3036198771 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3f5400700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f8070340 0x7fc3f8148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.324+0000 7fc3f5e00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f806dac0 0x7fc3f81f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3f6800700 1 -- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f806dac0 msgr2=0x7fc3f81f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3f6800700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f806dac0 0x7fc3f81f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3f6800700 1 -- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f8070340 msgr2=0x7fc3f8148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3f6800700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f8070340 0x7fc3f8148770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3f6800700 1 -- 172.21.15.3:0/3036198771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e40094b0 con 0x7fc3f81541b0 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3f6800700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f81541b0 0x7fc3f80ad900 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0007e70 tx=0x7fc3f000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3ee000700 1 -- 172.21.15.3:0/3036198771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f000ed30 con 0x7fc3f81541b0 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3ee000700 1 -- 172.21.15.3:0/3036198771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f000eeb0 con 0x7fc3f81541b0 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f815b860 con 0x7fc3f81541b0 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3ee000700 1 -- 172.21.15.3:0/3036198771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0014640 con 0x7fc3f81541b0 2024-09-13T16:51:31.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f815be20 con 0x7fc3f81541b0 2024-09-13T16:51:31.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.328+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f815d6a0 con 0x7fc3f81541b0 2024-09-13T16:51:31.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.332+0000 7fc3ee000700 1 -- 172.21.15.3:0/3036198771 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc3f00147a0 con 0x7fc3f81541b0 2024-09-13T16:51:31.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.332+0000 7fc3ee000700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc3d8066730 0x7fc3d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.332+0000 7fc3ee000700 1 -- 172.21.15.3:0/3036198771 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc3f0084c80 con 0x7fc3f81541b0 2024-09-13T16:51:31.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.332+0000 7fc3f5e00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc3d8066730 0x7fc3d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.332+0000 7fc3f5e00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc3d8066730 0x7fc3d8068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8000f60 tx=0x7fc3e8004a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.340+0000 7fc3ee000700 1 -- 172.21.15.3:0/3036198771 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc3f0051ba0 con 0x7fc3f81541b0 2024-09-13T16:51:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.524+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3f8070120 con 0x7fc3f81541b0 2024-09-13T16:51:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.524+0000 7fc3ee000700 1 -- 172.21.15.3:0/3036198771 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3f0084020 con 0x7fc3f81541b0 2024-09-13T16:51:31.530 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc3d8066730 msgr2=0x7fc3d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc3d8066730 0x7fc3d8068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8000f60 tx=0x7fc3e8004a70 comp rx=0 tx=0).stop 2024-09-13T16:51:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f81541b0 msgr2=0x7fc3f80ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f81541b0 0x7fc3f80ad900 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0007e70 tx=0x7fc3f000e3c0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 shutdown_connections 2024-09-13T16:51:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc3d8066730 0x7fc3d8068bf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3f81541b0 0x7fc3f80ad900 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f8070340 0x7fc3f8148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 --2- 172.21.15.3:0/3036198771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f806dac0 0x7fc3f81f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 >> 172.21.15.3:0/3036198771 conn(0x7fc3f806b280 msgr2=0x7fc3f80b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 shutdown_connections 2024-09-13T16:51:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.528+0000 7fc3fcc00700 1 -- 172.21.15.3:0/3036198771 wait complete. 2024-09-13T16:51:31.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:31.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.728+0000 7efd97e00700 1 Processor -- start 2024-09-13T16:51:31.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.728+0000 7efd97e00700 1 -- start start 2024-09-13T16:51:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd980cd7c0 0x7efd980cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd9815b110 0x7efd9815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd980b41e0 con 0x7efd981442c0 2024-09-13T16:51:31.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9806a390 con 0x7efd9815b110 2024-09-13T16:51:31.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9806b1f0 con 0x7efd980cd7c0 2024-09-13T16:51:31.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd9815b110 0x7efd9815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd9815b110 0x7efd9815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58716/0 (socket says 172.21.15.3:58716) 2024-09-13T16:51:31.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97400700 1 -- 172.21.15.3:0/3615442877 learned_addr learned my addr 172.21.15.3:0/3615442877 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:31.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd96000700 1 --2- 172.21.15.3:0/3615442877 >> v2:172.21.15.157:3300/0 conn(0x7efd980cd7c0 0x7efd980cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97400700 1 -- 172.21.15.3:0/3615442877 >> v2:172.21.15.157:3300/0 conn(0x7efd980cd7c0 msgr2=0x7efd980cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:31.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97400700 1 --2- 172.21.15.3:0/3615442877 >> v2:172.21.15.157:3300/0 conn(0x7efd980cd7c0 0x7efd980cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97400700 1 -- 172.21.15.3:0/3615442877 >> v1:172.21.15.157:6789/0 conn(0x7efd981442c0 legacy=0x7efd9806d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97400700 1 -- 172.21.15.3:0/3615442877 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980ce170 con 0x7efd9815b110 2024-09-13T16:51:31.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd96000700 1 --2- 172.21.15.3:0/3615442877 >> v2:172.21.15.157:3300/0 conn(0x7efd980cd7c0 0x7efd980cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:31.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd97400700 1 --2- 172.21.15.3:0/3615442877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd9815b110 0x7efd9815b580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efd980ce300 tx=0x7efd9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70438e030566d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:31.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd95600700 1 -- 172.21.15.3:0/3615442877 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9000e070 con 0x7efd9815b110 2024-09-13T16:51:31.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd95600700 1 -- 172.21.15.3:0/3615442877 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd90003a40 con 0x7efd9815b110 2024-09-13T16:51:31.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.732+0000 7efd95600700 1 -- 172.21.15.3:0/3615442877 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90003df0 con 0x7efd9815b110 2024-09-13T16:51:31.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- 172.21.15.3:0/3615442877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd9815b110 msgr2=0x7efd9815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 --2- 172.21.15.3:0/3615442877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd9815b110 0x7efd9815b580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efd980ce300 tx=0x7efd9000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:31.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- 172.21.15.3:0/3615442877 shutdown_connections 2024-09-13T16:51:31.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 --2- 172.21.15.3:0/3615442877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd9815b110 0x7efd9815b580 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 --2- 172.21.15.3:0/3615442877 >> v2:172.21.15.157:3300/0 conn(0x7efd980cd7c0 0x7efd980cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- 172.21.15.3:0/3615442877 >> 172.21.15.3:0/3615442877 conn(0x7efd980af620 msgr2=0x7efd980afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:31.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- 172.21.15.3:0/3615442877 shutdown_connections 2024-09-13T16:51:31.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- 172.21.15.3:0/3615442877 wait complete. 2024-09-13T16:51:31.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 Processor -- start 2024-09-13T16:51:31.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- start start 2024-09-13T16:51:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd980cd7c0 0x7efd9813b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd981442c0 0x7efd98071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9815b110 0x7efd9813cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9813bb50 con 0x7efd981442c0 2024-09-13T16:51:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815aa10 con 0x7efd980cd7c0 2024-09-13T16:51:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.736+0000 7efd97e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9813cb20 con 0x7efd9815b110 2024-09-13T16:51:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd981442c0 0x7efd98071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd981442c0 0x7efd98071180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58730/0 (socket says 172.21.15.3:58730) 2024-09-13T16:51:31.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 -- 172.21.15.3:0/3525447340 learned_addr learned my addr 172.21.15.3:0/3525447340 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96a00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd980cd7c0 0x7efd9813b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd97400700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9815b110 0x7efd9813cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 -- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9815b110 msgr2=0x7efd9813cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9815b110 0x7efd9813cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 -- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd980cd7c0 msgr2=0x7efd9813b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd980cd7c0 0x7efd9813b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 -- 172.21.15.3:0/3525447340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd90009b30 con 0x7efd981442c0 2024-09-13T16:51:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96a00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd980cd7c0 0x7efd9813b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd96000700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd981442c0 0x7efd98071180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd8400e790 tx=0x7efd8400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd8f400700 1 -- 172.21.15.3:0/3525447340 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84004230 con 0x7efd981442c0 2024-09-13T16:51:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd8f400700 1 -- 172.21.15.3:0/3525447340 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84004390 con 0x7efd981442c0 2024-09-13T16:51:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd8f400700 1 -- 172.21.15.3:0/3525447340 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd840036c0 con 0x7efd981442c0 2024-09-13T16:51:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9813bf50 con 0x7efd981442c0 2024-09-13T16:51:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.740+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9813c7b0 con 0x7efd981442c0 2024-09-13T16:51:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.744+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9813c340 con 0x7efd981442c0 2024-09-13T16:51:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.744+0000 7efd8f400700 1 -- 172.21.15.3:0/3525447340 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efd84003840 con 0x7efd981442c0 2024-09-13T16:51:31.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.744+0000 7efd8f400700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7efd78066730 0x7efd78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:31.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.744+0000 7efd8f400700 1 -- 172.21.15.3:0/3525447340 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efd84083ff0 con 0x7efd981442c0 2024-09-13T16:51:31.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.744+0000 7efd96a00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7efd78066730 0x7efd78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.748+0000 7efd96a00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7efd78066730 0x7efd78068bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd88000f60 tx=0x7efd88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:31.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.752+0000 7efd8f400700 1 -- 172.21.15.3:0/3525447340 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efd84050fa0 con 0x7efd981442c0 2024-09-13T16:51:31.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.932+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd9806d350 con 0x7efd981442c0 2024-09-13T16:51:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd8f400700 1 -- 172.21.15.3:0/3525447340 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd840545c0 con 0x7efd981442c0 2024-09-13T16:51:31.940 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7efd78066730 msgr2=0x7efd78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7efd78066730 0x7efd78068bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd88000f60 tx=0x7efd88009040 comp rx=0 tx=0).stop 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd981442c0 msgr2=0x7efd98071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd981442c0 0x7efd98071180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd8400e790 tx=0x7efd8400eb50 comp rx=0 tx=0).stop 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 shutdown_connections 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7efd78066730 0x7efd78068bf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9815b110 0x7efd9813cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd981442c0 0x7efd98071180 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 --2- 172.21.15.3:0/3525447340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd980cd7c0 0x7efd9813b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.936+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 >> 172.21.15.3:0/3525447340 conn(0x7efd980af620 msgr2=0x7efd980b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.940+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 shutdown_connections 2024-09-13T16:51:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:31.940+0000 7efd97e00700 1 -- 172.21.15.3:0/3525447340 wait complete. 2024-09-13T16:51:31.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:32.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.144+0000 7fb270a00700 1 Processor -- start 2024-09-13T16:51:32.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.144+0000 7fb270a00700 1 -- start start 2024-09-13T16:51:32.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb270a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c0b3aa0 0x7fb26c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb270a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb26c160900 0x7fb26c160d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb270a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c161760 con 0x7fb26c15ce20 2024-09-13T16:51:32.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb270a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c1612d0 con 0x7fb26c160900 2024-09-13T16:51:32.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb270a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c15d200 con 0x7fb26c0b3aa0 2024-09-13T16:51:32.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb26b200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb26c160900 0x7fb26c160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb269e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c0b3aa0 0x7fb26c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb26b200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb26c160900 0x7fb26c160d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58738/0 (socket says 172.21.15.3:58738) 2024-09-13T16:51:32.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb26a800700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb26c15ce20 0x7fb26c0cdd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:46836/0 (socket says 172.21.15.3:46836) 2024-09-13T16:51:32.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb26b200700 1 -- 172.21.15.3:0/1055923184 learned_addr learned my addr 172.21.15.3:0/1055923184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:32.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb269e00700 1 -- 172.21.15.3:0/1055923184 >> v2:172.21.15.3:3300/0 conn(0x7fb26c160900 msgr2=0x7fb26c160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:32.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb269e00700 1 --2- 172.21.15.3:0/1055923184 >> v2:172.21.15.3:3300/0 conn(0x7fb26c160900 0x7fb26c160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb269e00700 1 -- 172.21.15.3:0/1055923184 >> v1:172.21.15.157:6789/0 conn(0x7fb26c15ce20 legacy=0x7fb26c0cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb269e00700 1 -- 172.21.15.3:0/1055923184 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c0b2a00 con 0x7fb26c0b3aa0 2024-09-13T16:51:32.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.148+0000 7fb26b200700 1 --2- 172.21.15.3:0/1055923184 >> v2:172.21.15.3:3300/0 conn(0x7fb26c160900 0x7fb26c160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:32.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb269e00700 1 --2- 172.21.15.3:0/1055923184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c0b3aa0 0x7fb26c0b3f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb258009850 tx=0x7fb258009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50baf81182fe02f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:32.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb269400700 1 -- 172.21.15.3:0/1055923184 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258011830 con 0x7fb26c0b3aa0 2024-09-13T16:51:32.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb269400700 1 -- 172.21.15.3:0/1055923184 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb258011990 con 0x7fb26c0b3aa0 2024-09-13T16:51:32.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb269400700 1 -- 172.21.15.3:0/1055923184 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258011d40 con 0x7fb26c0b3aa0 2024-09-13T16:51:32.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb270a00700 1 -- 172.21.15.3:0/1055923184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c0b3aa0 msgr2=0x7fb26c0b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb270a00700 1 --2- 172.21.15.3:0/1055923184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c0b3aa0 0x7fb26c0b3f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb258009850 tx=0x7fb258009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:32.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb270a00700 1 -- 172.21.15.3:0/1055923184 shutdown_connections 2024-09-13T16:51:32.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb270a00700 1 --2- 172.21.15.3:0/1055923184 >> v2:172.21.15.3:3300/0 conn(0x7fb26c160900 0x7fb26c160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb270a00700 1 --2- 172.21.15.3:0/1055923184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c0b3aa0 0x7fb26c0b3f10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb270a00700 1 -- 172.21.15.3:0/1055923184 >> 172.21.15.3:0/1055923184 conn(0x7fb26c0d2f10 msgr2=0x7fb26c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:32.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb270a00700 1 -- 172.21.15.3:0/1055923184 shutdown_connections 2024-09-13T16:51:32.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.152+0000 7fb270a00700 1 -- 172.21.15.3:0/1055923184 wait complete. 2024-09-13T16:51:32.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 Processor -- start 2024-09-13T16:51:32.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 -- start start 2024-09-13T16:51:32.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c0b3aa0 0x7fb26c150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c15ce20 0x7fb26c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c160900 0x7fb26c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0cb800 con 0x7fb26c15ce20 2024-09-13T16:51:32.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c151070 con 0x7fb26c0b3aa0 2024-09-13T16:51:32.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0cb520 con 0x7fb26c160900 2024-09-13T16:51:32.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c15ce20 0x7fb26c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c15ce20 0x7fb26c0ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58740/0 (socket says 172.21.15.3:58740) 2024-09-13T16:51:32.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 -- 172.21.15.3:0/795187643 learned_addr learned my addr 172.21.15.3:0/795187643 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:32.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb26a800700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c0b3aa0 0x7fb26c150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb26b200700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c160900 0x7fb26c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 -- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c160900 msgr2=0x7fb26c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c160900 0x7fb26c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 -- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c0b3aa0 msgr2=0x7fb26c150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c0b3aa0 0x7fb26c150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 -- 172.21.15.3:0/795187643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb264009b30 con 0x7fb26c15ce20 2024-09-13T16:51:32.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb269e00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c15ce20 0x7fb26c0ccab0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb2580090b0 tx=0x7fb258004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:32.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb262a00700 1 -- 172.21.15.3:0/795187643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258004570 con 0x7fb26c15ce20 2024-09-13T16:51:32.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb262a00700 1 -- 172.21.15.3:0/795187643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25801f070 con 0x7fb26c15ce20 2024-09-13T16:51:32.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2580094b0 con 0x7fb26c15ce20 2024-09-13T16:51:32.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.156+0000 7fb262a00700 1 -- 172.21.15.3:0/795187643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258003900 con 0x7fb26c15ce20 2024-09-13T16:51:32.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.160+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c0a9b90 con 0x7fb26c15ce20 2024-09-13T16:51:32.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.160+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c0a9710 con 0x7fb26c15ce20 2024-09-13T16:51:32.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.160+0000 7fb262a00700 1 -- 172.21.15.3:0/795187643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb258021030 con 0x7fb26c15ce20 2024-09-13T16:51:32.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.164+0000 7fb262a00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb24c066760 0x7fb24c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.164+0000 7fb262a00700 1 -- 172.21.15.3:0/795187643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb258085910 con 0x7fb26c15ce20 2024-09-13T16:51:32.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.164+0000 7fb26a800700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb24c066760 0x7fb24c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.164+0000 7fb26a800700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb24c066760 0x7fb24c068c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb25c000f60 tx=0x7fb25c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.168+0000 7fb262a00700 1 -- 172.21.15.3:0/795187643 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb26c0a9710 con 0x7fb26c15ce20 2024-09-13T16:51:32.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.348+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb26c0a9710 con 0x7fb26c15ce20 2024-09-13T16:51:32.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.348+0000 7fb262a00700 1 -- 172.21.15.3:0/795187643 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb26c0a9710 con 0x7fb26c15ce20 2024-09-13T16:51:32.354 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb24c066760 msgr2=0x7fb24c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb24c066760 0x7fb24c068c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb25c000f60 tx=0x7fb25c007d30 comp rx=0 tx=0).stop 2024-09-13T16:51:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c15ce20 msgr2=0x7fb26c0ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c15ce20 0x7fb26c0ccab0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb2580090b0 tx=0x7fb258004770 comp rx=0 tx=0).stop 2024-09-13T16:51:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 shutdown_connections 2024-09-13T16:51:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb24c066760 0x7fb24c068c20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c160900 0x7fb26c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c15ce20 0x7fb26c0ccab0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 --2- 172.21.15.3:0/795187643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c0b3aa0 0x7fb26c150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 >> 172.21.15.3:0/795187643 conn(0x7fb26c0d2f10 msgr2=0x7fb26c0b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 shutdown_connections 2024-09-13T16:51:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.352+0000 7fb270a00700 1 -- 172.21.15.3:0/795187643 wait complete. 2024-09-13T16:51:32.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj' 2024-09-13T16:51:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.544+0000 7f6d4e400700 1 Processor -- start 2024-09-13T16:51:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.544+0000 7f6d4e400700 1 -- start start 2024-09-13T16:51:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.544+0000 7f6d4e400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d480cc6d0 0x7f6d480b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.544+0000 7f6d4e400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d481595f0 con 0x7f6d48156be0 2024-09-13T16:51:32.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.544+0000 7f6d4e400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d481598d0 con 0x7f6d4806b7a0 2024-09-13T16:51:32.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.544+0000 7f6d4e400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4815a270 con 0x7f6d480cc6d0 2024-09-13T16:51:32.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.544+0000 7f6d4d000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d48156be0 0x7f6d4806b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51626/0 (socket says 172.21.15.3:51626) 2024-09-13T16:51:32.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.544+0000 7f6d4d000700 1 -- 172.21.15.3:0/232683110 learned_addr learned my addr 172.21.15.3:0/232683110 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:32.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4da00700 1 --2- 172.21.15.3:0/232683110 >> v2:172.21.15.3:3300/0 conn(0x7f6d480cc6d0 0x7f6d480b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d47e00700 1 -- 172.21.15.3:0/232683110 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566567264 0 0) 0x7f6d481595f0 con 0x7f6d48156be0 2024-09-13T16:51:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d47e00700 1 -- 172.21.15.3:0/232683110 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d2c0035d0 con 0x7f6d48156be0 2024-09-13T16:51:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d47e00700 1 -- 172.21.15.3:0/232683110 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300015200 0 0) 0x7f6d481598d0 con 0x7f6d4806b7a0 2024-09-13T16:51:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d47e00700 1 -- 172.21.15.3:0/232683110 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d481595f0 con 0x7f6d4806b7a0 2024-09-13T16:51:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4da00700 1 -- 172.21.15.3:0/232683110 >> v1:172.21.15.157:6789/0 conn(0x7f6d4806b7a0 legacy=0x7f6d480b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4da00700 1 -- 172.21.15.3:0/232683110 >> v1:172.21.15.3:6789/0 conn(0x7f6d48156be0 legacy=0x7f6d4806b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4da00700 1 -- 172.21.15.3:0/232683110 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d48159bb0 con 0x7f6d480cc6d0 2024-09-13T16:51:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4da00700 1 --2- 172.21.15.3:0/232683110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d480cc6d0 0x7f6d480b4640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009d90 tx=0x7f6d4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6d1f55b8962bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d47e00700 1 -- 172.21.15.3:0/232683110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000e070 con 0x7f6d480cc6d0 2024-09-13T16:51:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d47e00700 1 -- 172.21.15.3:0/232683110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4000bb20 con 0x7f6d480cc6d0 2024-09-13T16:51:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d47e00700 1 -- 172.21.15.3:0/232683110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000be90 con 0x7f6d480cc6d0 2024-09-13T16:51:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4e400700 1 -- 172.21.15.3:0/232683110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d480cc6d0 msgr2=0x7f6d480b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4e400700 1 --2- 172.21.15.3:0/232683110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d480cc6d0 0x7f6d480b4640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009d90 tx=0x7f6d4000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4e400700 1 -- 172.21.15.3:0/232683110 shutdown_connections 2024-09-13T16:51:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4e400700 1 --2- 172.21.15.3:0/232683110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d480cc6d0 0x7f6d480b4640 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4e400700 1 -- 172.21.15.3:0/232683110 >> 172.21.15.3:0/232683110 conn(0x7f6d4806a570 msgr2=0x7f6d4806a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4e400700 1 -- 172.21.15.3:0/232683110 shutdown_connections 2024-09-13T16:51:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.548+0000 7f6d4e400700 1 -- 172.21.15.3:0/232683110 wait complete. 2024-09-13T16:51:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.552+0000 7f6d4e400700 1 Processor -- start 2024-09-13T16:51:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.552+0000 7f6d4e400700 1 -- start start 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4e400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4806b7a0 0x7f6d4814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4e400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d480cc6d0 0x7f6d481eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4e400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d48156be0 0x7f6d480af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4e400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d480ccb80 con 0x7f6d48156be0 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4e400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d481ef510 con 0x7f6d4806b7a0 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4e400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d48153130 con 0x7f6d480cc6d0 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d48156be0 0x7f6d480af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d48156be0 0x7f6d480af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58758/0 (socket says 172.21.15.3:58758) 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 -- 172.21.15.3:0/3270329300 learned_addr learned my addr 172.21.15.3:0/3270329300 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4d000700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4806b7a0 0x7f6d4814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d3fe00700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d480cc6d0 0x7f6d481eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 -- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d480cc6d0 msgr2=0x7f6d481eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d480cc6d0 0x7f6d481eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 -- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4806b7a0 msgr2=0x7f6d4814dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4806b7a0 0x7f6d4814dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 -- 172.21.15.3:0/3270329300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40009a70 con 0x7f6d48156be0 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d3fe00700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d480cc6d0 0x7f6d481eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4da00700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d48156be0 0x7f6d480af4d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000b680 tx=0x7f6d40004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d45600700 1 -- 172.21.15.3:0/3270329300 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40004960 con 0x7f6d48156be0 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d45600700 1 -- 172.21.15.3:0/3270329300 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d40020070 con 0x7f6d48156be0 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d45600700 1 -- 172.21.15.3:0/3270329300 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400276e0 con 0x7f6d48156be0 2024-09-13T16:51:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d481f1f70 con 0x7f6d48156be0 2024-09-13T16:51:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.556+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d481f2530 con 0x7f6d48156be0 2024-09-13T16:51:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.560+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d481532a0 con 0x7f6d48156be0 2024-09-13T16:51:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.560+0000 7f6d45600700 1 -- 172.21.15.3:0/3270329300 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d40027840 con 0x7f6d48156be0 2024-09-13T16:51:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.564+0000 7f6d45600700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d24066760 0x7f6d24068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.564+0000 7f6d45600700 1 -- 172.21.15.3:0/3270329300 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6d40022070 con 0x7f6d48156be0 2024-09-13T16:51:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.564+0000 7f6d4d000700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d24066760 0x7f6d24068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:32.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.564+0000 7f6d4d000700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d24066760 0x7f6d24068c20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d38000f60 tx=0x7f6d38007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:32.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.568+0000 7f6d45600700 1 -- 172.21.15.3:0/3270329300 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6d481532a0 con 0x7f6d48156be0 2024-09-13T16:51:32.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:32.816+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]} v 0) v1 -- 0x7f6d481532a0 con 0x7f6d48156be0 2024-09-13T16:51:33.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.368+0000 7f6d45600700 1 -- 172.21.15.3:0/3270329300 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]}]=0 v78) v1 ==== 150+0+0 (secure 0 0 0) 0x7f6d481532a0 con 0x7f6d48156be0 2024-09-13T16:51:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d24066760 msgr2=0x7f6d24068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d24066760 0x7f6d24068c20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d38000f60 tx=0x7f6d38007e20 comp rx=0 tx=0).stop 2024-09-13T16:51:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d48156be0 msgr2=0x7f6d480af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d48156be0 0x7f6d480af4d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000b680 tx=0x7f6d40004af0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 shutdown_connections 2024-09-13T16:51:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6d24066760 0x7f6d24068c20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d48156be0 0x7f6d480af4d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d480cc6d0 0x7f6d481eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 --2- 172.21.15.3:0/3270329300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4806b7a0 0x7f6d4814dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 >> 172.21.15.3:0/3270329300 conn(0x7f6d4806a570 msgr2=0x7f6d4806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 shutdown_connections 2024-09-13T16:51:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.372+0000 7f6d4e400700 1 -- 172.21.15.3:0/3270329300 wait complete. 2024-09-13T16:51:33.403 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T16:51:33.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f09a5200700 1 Processor -- start 2024-09-13T16:51:33.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f09a5200700 1 -- start start 2024-09-13T16:51:33.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f09a5200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09a01442c0 0x7f09a006d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:33.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f09a5200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a006dcb0 0x7f09a00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:33.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f09a5200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09a00cdd00 0x7f09a00b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:33.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f09a5200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a006e0f0 con 0x7f09a00cdd00 2024-09-13T16:51:33.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f09a5200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a00b41e0 con 0x7f09a01442c0 2024-09-13T16:51:33.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f09a5200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a006a390 con 0x7f09a006dcb0 2024-09-13T16:51:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f099fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09a00cdd00 0x7f09a00b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f0996a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a006dcb0 0x7f09a00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f099f400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09a01442c0 0x7f09a006d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f0996a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a006dcb0 0x7f09a00cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:34818/0 (socket says 172.21.15.3:34818) 2024-09-13T16:51:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.572+0000 7f0996a00700 1 -- 172.21.15.3:0/2159100154 learned_addr learned my addr 172.21.15.3:0/2159100154 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:33.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f0996a00700 1 -- 172.21.15.3:0/2159100154 >> v2:172.21.15.157:3300/0 conn(0x7f09a01442c0 msgr2=0x7f09a006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:33.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f0996a00700 1 --2- 172.21.15.3:0/2159100154 >> v2:172.21.15.157:3300/0 conn(0x7f09a01442c0 0x7f09a006d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f0996a00700 1 -- 172.21.15.3:0/2159100154 >> v2:172.21.15.3:3300/0 conn(0x7f09a00cdd00 msgr2=0x7f09a00b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:33.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f0996a00700 1 --2- 172.21.15.3:0/2159100154 >> v2:172.21.15.3:3300/0 conn(0x7f09a00cdd00 0x7f09a00b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f0996a00700 1 -- 172.21.15.3:0/2159100154 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a006b1f0 con 0x7f09a006dcb0 2024-09-13T16:51:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f099fe00700 1 --2- 172.21.15.3:0/2159100154 >> v2:172.21.15.3:3300/0 conn(0x7f09a00cdd00 0x7f09a00b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f0996a00700 1 --2- 172.21.15.3:0/2159100154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a006dcb0 0x7f09a00cd7c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09a0066e70 tx=0x7f0988009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94aec08ce70e33d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f099ea00700 1 -- 172.21.15.3:0/2159100154 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988011830 con 0x7f09a006dcb0 2024-09-13T16:51:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f099ea00700 1 -- 172.21.15.3:0/2159100154 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988011990 con 0x7f09a006dcb0 2024-09-13T16:51:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f099ea00700 1 -- 172.21.15.3:0/2159100154 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988011d20 con 0x7f09a006dcb0 2024-09-13T16:51:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 -- 172.21.15.3:0/2159100154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a006dcb0 msgr2=0x7f09a00cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 --2- 172.21.15.3:0/2159100154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a006dcb0 0x7f09a00cd7c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09a0066e70 tx=0x7f0988009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 -- 172.21.15.3:0/2159100154 shutdown_connections 2024-09-13T16:51:33.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 --2- 172.21.15.3:0/2159100154 >> v2:172.21.15.3:3300/0 conn(0x7f09a00cdd00 0x7f09a00b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 --2- 172.21.15.3:0/2159100154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a006dcb0 0x7f09a00cd7c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 --2- 172.21.15.3:0/2159100154 >> v2:172.21.15.157:3300/0 conn(0x7f09a01442c0 0x7f09a006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 -- 172.21.15.3:0/2159100154 >> 172.21.15.3:0/2159100154 conn(0x7f09a00af620 msgr2=0x7f09a00afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:33.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 -- 172.21.15.3:0/2159100154 shutdown_connections 2024-09-13T16:51:33.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.576+0000 7f09a5200700 1 -- 172.21.15.3:0/2159100154 wait complete. 2024-09-13T16:51:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f09a5200700 1 Processor -- start 2024-09-13T16:51:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f09a5200700 1 -- start start 2024-09-13T16:51:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f09a5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a006dcb0 0x7f09a01592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f09a5200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a00cdd00 0x7f09a01e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f09a5200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01442c0 0x7f09a01e6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:33.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f09a5200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a0141020 con 0x7f09a01442c0 2024-09-13T16:51:33.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f09a5200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a013a0c0 con 0x7f09a006dcb0 2024-09-13T16:51:33.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f09a5200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a01e3230 con 0x7f09a00cdd00 2024-09-13T16:51:33.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f099fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01442c0 0x7f09a01e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f099fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01442c0 0x7f09a01e6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58776/0 (socket says 172.21.15.3:58776) 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f099fe00700 1 -- 172.21.15.3:0/2010133495 learned_addr learned my addr 172.21.15.3:0/2010133495 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f0996a00700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a00cdd00 0x7f09a01e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.580+0000 7f099f400700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a006dcb0 0x7f09a01592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f099fe00700 1 -- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a00cdd00 msgr2=0x7f09a01e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f099fe00700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a00cdd00 0x7f09a01e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f099fe00700 1 -- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a006dcb0 msgr2=0x7f09a01592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f099fe00700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a006dcb0 0x7f09a01592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f099fe00700 1 -- 172.21.15.3:0/2010133495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09980094b0 con 0x7f09a01442c0 2024-09-13T16:51:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f099f400700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a006dcb0 0x7f09a01592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f099fe00700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01442c0 0x7f09a01e6300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f099800e980 tx=0x7f099800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f0997e00700 1 -- 172.21.15.3:0/2010133495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800eed0 con 0x7f09a01442c0 2024-09-13T16:51:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f0997e00700 1 -- 172.21.15.3:0/2010133495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0998004020 con 0x7f09a01442c0 2024-09-13T16:51:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f0997e00700 1 -- 172.21.15.3:0/2010133495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980043b0 con 0x7f09a01442c0 2024-09-13T16:51:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0988009570 con 0x7f09a01442c0 2024-09-13T16:51:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a01e3480 con 0x7f09a01442c0 2024-09-13T16:51:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.584+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a01446a0 con 0x7f09a01442c0 2024-09-13T16:51:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.588+0000 7f0997e00700 1 -- 172.21.15.3:0/2010133495 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f099800c850 con 0x7f09a01442c0 2024-09-13T16:51:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.588+0000 7f0997e00700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f097c066730 0x7f097c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.588+0000 7f0997e00700 1 -- 172.21.15.3:0/2010133495 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0998084310 con 0x7f09a01442c0 2024-09-13T16:51:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.588+0000 7f099f400700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f097c066730 0x7f097c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:33.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.588+0000 7f099f400700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f097c066730 0x7f097c068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0990009fd0 tx=0x7f099000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:33.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.592+0000 7f0997e00700 1 -- 172.21.15.3:0/2010133495 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0998051190 con 0x7f09a01442c0 2024-09-13T16:51:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.772+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f09a00b4670 con 0x7f09a01442c0 2024-09-13T16:51:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.780+0000 7f0997e00700 1 -- 172.21.15.3:0/2010133495 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f09980547b0 con 0x7f09a01442c0 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f097c066730 msgr2=0x7f097c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f097c066730 0x7f097c068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0990009fd0 tx=0x7f099000b040 comp rx=0 tx=0).stop 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01442c0 msgr2=0x7f09a01e6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01442c0 0x7f09a01e6300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f099800e980 tx=0x7f099800ec90 comp rx=0 tx=0).stop 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 shutdown_connections 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f097c066730 0x7f097c068bf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01442c0 0x7f09a01e6300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a00cdd00 0x7f09a01e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 --2- 172.21.15.3:0/2010133495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a006dcb0 0x7f09a01592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 >> 172.21.15.3:0/2010133495 conn(0x7f09a00af620 msgr2=0x7f09a00b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 shutdown_connections 2024-09-13T16:51:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:33.784+0000 7f09a5200700 1 -- 172.21.15.3:0/2010133495 wait complete. 2024-09-13T16:51:33.812 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T16:51:33.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T16:51:33.847 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T16:51:33.890 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T16:51:33.890 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:51:33.890 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:33.915 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:51:33.916 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:33.979 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:51:33.979 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:34.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:34.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f8531800700 1 Processor -- start 2024-09-13T16:51:34.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f8531800700 1 -- start start 2024-09-13T16:51:34.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f8531800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f8531800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c0ce8e0 con 0x7f852c06ccf0 2024-09-13T16:51:34.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f8531800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c0ccd30 con 0x7f852c144790 2024-09-13T16:51:34.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f8531800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0cd010 con 0x7f852c1443b0 2024-09-13T16:51:34.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f8530e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f852c06ccf0 0x7f852c15b220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37392/0 (socket says 172.21.15.3:37392) 2024-09-13T16:51:34.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f8530e00700 1 -- 172.21.15.3:0/1131649449 learned_addr learned my addr 172.21.15.3:0/1131649449 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:34.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.256+0000 7f852be00700 1 --2- 172.21.15.3:0/1131649449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852b400700 1 -- 172.21.15.3:0/1131649449 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123009099 0 0) 0x7f852c0ce8e0 con 0x7f852c06ccf0 2024-09-13T16:51:34.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852b400700 1 -- 172.21.15.3:0/1131649449 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85100035d0 con 0x7f852c06ccf0 2024-09-13T16:51:34.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852b400700 1 -- 172.21.15.3:0/1131649449 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105245294 0 0) 0x7f852c0ccd30 con 0x7f852c144790 2024-09-13T16:51:34.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852b400700 1 -- 172.21.15.3:0/1131649449 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f852c0ce8e0 con 0x7f852c144790 2024-09-13T16:51:34.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852be00700 1 -- 172.21.15.3:0/1131649449 >> v1:172.21.15.157:6789/0 conn(0x7f852c144790 legacy=0x7f852c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852be00700 1 -- 172.21.15.3:0/1131649449 >> v1:172.21.15.3:6789/0 conn(0x7f852c06ccf0 legacy=0x7f852c15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852be00700 1 -- 172.21.15.3:0/1131649449 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0cd2f0 con 0x7f852c1443b0 2024-09-13T16:51:34.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852be00700 1 --2- 172.21.15.3:0/1131649449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c0b39f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f851c009880 tx=0x7f851c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25959db3ecf35be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:34.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852b400700 1 -- 172.21.15.3:0/1131649449 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c011830 con 0x7f852c1443b0 2024-09-13T16:51:34.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852b400700 1 -- 172.21.15.3:0/1131649449 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c011990 con 0x7f852c1443b0 2024-09-13T16:51:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.260+0000 7f852b400700 1 -- 172.21.15.3:0/1131649449 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c011d20 con 0x7f852c1443b0 2024-09-13T16:51:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 -- 172.21.15.3:0/1131649449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 msgr2=0x7f852c0b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 --2- 172.21.15.3:0/1131649449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c0b39f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f851c009880 tx=0x7f851c009b90 comp rx=0 tx=0).stop 2024-09-13T16:51:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 -- 172.21.15.3:0/1131649449 shutdown_connections 2024-09-13T16:51:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 --2- 172.21.15.3:0/1131649449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c0b39f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 -- 172.21.15.3:0/1131649449 >> 172.21.15.3:0/1131649449 conn(0x7f852c0b0cf0 msgr2=0x7f852c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 -- 172.21.15.3:0/1131649449 shutdown_connections 2024-09-13T16:51:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 -- 172.21.15.3:0/1131649449 wait complete. 2024-09-13T16:51:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 Processor -- start 2024-09-13T16:51:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.264+0000 7f8531800700 1 -- start start 2024-09-13T16:51:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8531800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f852c06ccf0 0x7f852c13cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8531800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8531800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c14ea40 0x7f852c13d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8531800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c13e290 con 0x7f852c14ea40 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8531800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c13dfb0 con 0x7f852c06ccf0 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8531800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c13c050 con 0x7f852c1443b0 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c14ea40 0x7f852c13d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c14ea40 0x7f852c13d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40424/0 (socket says 172.21.15.3:40424) 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 -- 172.21.15.3:0/438069830 learned_addr learned my addr 172.21.15.3:0/438069830 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f852be00700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f852c06ccf0 0x7f852c13cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8523400700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 -- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 msgr2=0x7f852c152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 -- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f852c06ccf0 msgr2=0x7f852c13cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f852c06ccf0 0x7f852c13cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 -- 172.21.15.3:0/438069830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0094b0 con 0x7f852c14ea40 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f852be00700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f852c06ccf0 0x7f852c13cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8523400700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8530e00700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c14ea40 0x7f852c13d640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f852400cba0 tx=0x7f852400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8528c00700 1 -- 172.21.15.3:0/438069830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400a980 con 0x7f852c14ea40 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8528c00700 1 -- 172.21.15.3:0/438069830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852400aae0 con 0x7f852c14ea40 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c13c330 con 0x7f852c14ea40 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8528c00700 1 -- 172.21.15.3:0/438069830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400ae70 con 0x7f852c14ea40 2024-09-13T16:51:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.268+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c152c50 con 0x7f852c14ea40 2024-09-13T16:51:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.272+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c13be90 con 0x7f852c14ea40 2024-09-13T16:51:34.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.272+0000 7f8528c00700 1 -- 172.21.15.3:0/438069830 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f852401b3b0 con 0x7f852c14ea40 2024-09-13T16:51:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.276+0000 7f8528c00700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8508066730 0x7f8508068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.276+0000 7f8528c00700 1 -- 172.21.15.3:0/438069830 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8524083ff0 con 0x7f852c14ea40 2024-09-13T16:51:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.276+0000 7f852be00700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8508066730 0x7f8508068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.276+0000 7f852be00700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8508066730 0x7f8508068bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f851c0090b0 tx=0x7f851c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.280+0000 7f8528c00700 1 -- 172.21.15.3:0/438069830 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8524050fa0 con 0x7f852c14ea40 2024-09-13T16:51:34.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.472+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f852c0ccd30 con 0x7f852c14ea40 2024-09-13T16:51:34.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.472+0000 7f8528c00700 1 -- 172.21.15.3:0/438069830 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f85240545c0 con 0x7f852c14ea40 2024-09-13T16:51:34.480 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:34.480 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:34.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8508066730 msgr2=0x7f8508068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8508066730 0x7f8508068bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f851c0090b0 tx=0x7f851c01c040 comp rx=0 tx=0).stop 2024-09-13T16:51:34.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c14ea40 msgr2=0x7f852c13d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c14ea40 0x7f852c13d640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f852400cba0 tx=0x7f852400cc80 comp rx=0 tx=0).stop 2024-09-13T16:51:34.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 shutdown_connections 2024-09-13T16:51:34.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8508066730 0x7f8508068bf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c14ea40 0x7f852c13d640 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f852c1443b0 0x7f852c152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 --2- 172.21.15.3:0/438069830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f852c06ccf0 0x7f852c13cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 >> 172.21.15.3:0/438069830 conn(0x7f852c0b0cf0 msgr2=0x7f852c06fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:34.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 shutdown_connections 2024-09-13T16:51:34.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.480+0000 7f8531800700 1 -- 172.21.15.3:0/438069830 wait complete. 2024-09-13T16:51:34.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:34.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:34.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.688+0000 7fa220a00700 1 Processor -- start 2024-09-13T16:51:34.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.688+0000 7fa220a00700 1 -- start start 2024-09-13T16:51:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa220a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa21c0cefa0 0x7fa21c06e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa220a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa21c06c3b0 0x7fa21c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa220a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c06ec10 con 0x7fa21c0cebc0 2024-09-13T16:51:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa220a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c0b4640 con 0x7fa21c0cefa0 2024-09-13T16:51:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa220a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c0b4920 con 0x7fa21c06c3b0 2024-09-13T16:51:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21aa00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa21c0cefa0 0x7fa21c06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21be00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa21c06c3b0 0x7fa21c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21be00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa21c06c3b0 0x7fa21c06c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:57644/0 (socket says 172.21.15.3:57644) 2024-09-13T16:51:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21aa00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa21c0cefa0 0x7fa21c06e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40434/0 (socket says 172.21.15.3:40434) 2024-09-13T16:51:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21be00700 1 -- 172.21.15.3:0/2916189211 learned_addr learned my addr 172.21.15.3:0/2916189211 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21be00700 1 -- 172.21.15.3:0/2916189211 >> v2:172.21.15.3:3300/0 conn(0x7fa21c0cefa0 msgr2=0x7fa21c06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21be00700 1 --2- 172.21.15.3:0/2916189211 >> v2:172.21.15.3:3300/0 conn(0x7fa21c0cefa0 0x7fa21c06e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21be00700 1 -- 172.21.15.3:0/2916189211 >> v1:172.21.15.157:6789/0 conn(0x7fa21c0cebc0 legacy=0x7fa21c15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21be00700 1 -- 172.21.15.3:0/2916189211 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0b4c00 con 0x7fa21c06c3b0 2024-09-13T16:51:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21be00700 1 --2- 172.21.15.3:0/2916189211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c06c3b0 0x7fa21c06c7e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0b5fb0 tx=0x7fa21400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f58026f98b5873 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21a000700 1 -- 172.21.15.3:0/2916189211 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400e070 con 0x7fa21c06c3b0 2024-09-13T16:51:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21a000700 1 -- 172.21.15.3:0/2916189211 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21400b910 con 0x7fa21c06c3b0 2024-09-13T16:51:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.692+0000 7fa21a000700 1 -- 172.21.15.3:0/2916189211 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400bcc0 con 0x7fa21c06c3b0 2024-09-13T16:51:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 -- 172.21.15.3:0/2916189211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c06c3b0 msgr2=0x7fa21c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 --2- 172.21.15.3:0/2916189211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c06c3b0 0x7fa21c06c7e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0b5fb0 tx=0x7fa21400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 -- 172.21.15.3:0/2916189211 shutdown_connections 2024-09-13T16:51:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 --2- 172.21.15.3:0/2916189211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c06c3b0 0x7fa21c06c7e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 --2- 172.21.15.3:0/2916189211 >> v2:172.21.15.3:3300/0 conn(0x7fa21c0cefa0 0x7fa21c06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 -- 172.21.15.3:0/2916189211 >> 172.21.15.3:0/2916189211 conn(0x7fa21c071da0 msgr2=0x7fa21c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:34.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 -- 172.21.15.3:0/2916189211 shutdown_connections 2024-09-13T16:51:34.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 -- 172.21.15.3:0/2916189211 wait complete. 2024-09-13T16:51:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.696+0000 7fa220a00700 1 Processor -- start 2024-09-13T16:51:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa220a00700 1 -- start start 2024-09-13T16:51:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa220a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa21c06c3b0 0x7fa21c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa220a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa21c0cebc0 0x7fa21c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa220a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c0cefa0 0x7fa21c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa220a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0aa130 con 0x7fa21c0cebc0 2024-09-13T16:51:34.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa220a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c15bc70 con 0x7fa21c0cefa0 2024-09-13T16:51:34.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa220a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c15bf50 con 0x7fa21c06c3b0 2024-09-13T16:51:34.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa21c0cebc0 0x7fa21c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa21c0cebc0 0x7fa21c15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40446/0 (socket says 172.21.15.3:40446) 2024-09-13T16:51:34.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21aa00700 1 -- 172.21.15.3:0/198682551 learned_addr learned my addr 172.21.15.3:0/198682551 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:34.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21b400700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa21c06c3b0 0x7fa21c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21be00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c0cefa0 0x7fa21c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21aa00700 1 -- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa21c06c3b0 msgr2=0x7fa21c15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21aa00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa21c06c3b0 0x7fa21c15d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21aa00700 1 -- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c0cefa0 msgr2=0x7fa21c15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21aa00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c0cefa0 0x7fa21c15b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.700+0000 7fa21aa00700 1 -- 172.21.15.3:0/198682551 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa214009b30 con 0x7fa21c0cebc0 2024-09-13T16:51:34.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.704+0000 7fa21aa00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa21c0cebc0 0x7fa21c15b090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa20800e840 tx=0x7fa20800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:34.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.704+0000 7fa213400700 1 -- 172.21.15.3:0/198682551 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080095a0 con 0x7fa21c0cebc0 2024-09-13T16:51:34.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.704+0000 7fa213400700 1 -- 172.21.15.3:0/198682551 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208013680 con 0x7fa21c0cebc0 2024-09-13T16:51:34.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.704+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c15c230 con 0x7fa21c0cebc0 2024-09-13T16:51:34.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.704+0000 7fa213400700 1 -- 172.21.15.3:0/198682551 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080139f0 con 0x7fa21c0cebc0 2024-09-13T16:51:34.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.704+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c15c7f0 con 0x7fa21c0cebc0 2024-09-13T16:51:34.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.704+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c06d0b0 con 0x7fa21c0cebc0 2024-09-13T16:51:34.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.708+0000 7fa213400700 1 -- 172.21.15.3:0/198682551 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa208013b50 con 0x7fa21c0cebc0 2024-09-13T16:51:34.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.708+0000 7fa213400700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa1fc066730 0x7fa1fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:34.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.708+0000 7fa213400700 1 -- 172.21.15.3:0/198682551 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa2080844a0 con 0x7fa21c0cebc0 2024-09-13T16:51:34.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.708+0000 7fa21b400700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa1fc066730 0x7fa1fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:34.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.712+0000 7fa21b400700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa1fc066730 0x7fa1fc068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa20c000f60 tx=0x7fa20c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:34.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.716+0000 7fa213400700 1 -- 172.21.15.3:0/198682551 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa2080513b0 con 0x7fa21c0cebc0 2024-09-13T16:51:34.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.896+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa21c140bf0 con 0x7fa21c0cebc0 2024-09-13T16:51:34.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.896+0000 7fa213400700 1 -- 172.21.15.3:0/198682551 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2080549d0 con 0x7fa21c0cebc0 2024-09-13T16:51:34.903 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.900+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa1fc066730 msgr2=0x7fa1fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.900+0000 7fa220a00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa1fc066730 0x7fa1fc068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa20c000f60 tx=0x7fa20c009040 comp rx=0 tx=0).stop 2024-09-13T16:51:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.900+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa21c0cebc0 msgr2=0x7fa21c15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.900+0000 7fa220a00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa21c0cebc0 0x7fa21c15b090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa20800e840 tx=0x7fa20800eb50 comp rx=0 tx=0).stop 2024-09-13T16:51:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.904+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 shutdown_connections 2024-09-13T16:51:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.904+0000 7fa220a00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa1fc066730 0x7fa1fc068bf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.904+0000 7fa220a00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21c0cefa0 0x7fa21c15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.904+0000 7fa220a00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa21c0cebc0 0x7fa21c15b090 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.904+0000 7fa220a00700 1 --2- 172.21.15.3:0/198682551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa21c06c3b0 0x7fa21c15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.904+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 >> 172.21.15.3:0/198682551 conn(0x7fa21c071da0 msgr2=0x7fa21c13d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:34.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.904+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 shutdown_connections 2024-09-13T16:51:34.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:34.904+0000 7fa220a00700 1 -- 172.21.15.3:0/198682551 wait complete. 2024-09-13T16:51:34.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.100+0000 7f1579400700 1 Processor -- start 2024-09-13T16:51:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.100+0000 7f1579400700 1 -- start start 2024-09-13T16:51:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1579400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15740ce5e0 0x7f1574163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1579400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1574159550 con 0x7f1574153450 2024-09-13T16:51:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1579400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15741597d0 con 0x7f1574153830 2024-09-13T16:51:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1579400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1574159ab0 con 0x7f15740ce5e0 2024-09-13T16:51:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1573e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1574153450 0x7f157406a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37400/0 (socket says 172.21.15.3:37400) 2024-09-13T16:51:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1573e00700 1 -- 172.21.15.3:0/226337128 learned_addr learned my addr 172.21.15.3:0/226337128 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1578a00700 1 --2- 172.21.15.3:0/226337128 >> v2:172.21.15.3:3300/0 conn(0x7f15740ce5e0 0x7f1574163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1573400700 1 -- 172.21.15.3:0/226337128 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782694427 0 0) 0x7f1574159550 con 0x7f1574153450 2024-09-13T16:51:35.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1573400700 1 -- 172.21.15.3:0/226337128 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15580035d0 con 0x7f1574153450 2024-09-13T16:51:35.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1578a00700 1 -- 172.21.15.3:0/226337128 >> v1:172.21.15.157:6789/0 conn(0x7f1574153830 legacy=0x7f15740b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1578a00700 1 -- 172.21.15.3:0/226337128 >> v1:172.21.15.3:6789/0 conn(0x7f1574153450 legacy=0x7f157406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1573400700 1 -- 172.21.15.3:0/226337128 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1225751149 0 0) 0x7f15580035d0 con 0x7f1574153450 2024-09-13T16:51:35.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.104+0000 7f1578a00700 1 -- 172.21.15.3:0/226337128 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574159d90 con 0x7f15740ce5e0 2024-09-13T16:51:35.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1578a00700 1 --2- 172.21.15.3:0/226337128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f1574163ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f156c009850 tx=0x7f156c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2859aa290bfed78a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1573400700 1 -- 172.21.15.3:0/226337128 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640019f0 con 0x7f15740ce5e0 2024-09-13T16:51:35.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1573400700 1 -- 172.21.15.3:0/226337128 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c004030 con 0x7f15740ce5e0 2024-09-13T16:51:35.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1573400700 1 -- 172.21.15.3:0/226337128 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c0043c0 con 0x7f15740ce5e0 2024-09-13T16:51:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1579400700 1 -- 172.21.15.3:0/226337128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 msgr2=0x7f1574163ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1579400700 1 --2- 172.21.15.3:0/226337128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f1574163ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f156c009850 tx=0x7f156c009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1579400700 1 -- 172.21.15.3:0/226337128 shutdown_connections 2024-09-13T16:51:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1579400700 1 --2- 172.21.15.3:0/226337128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f1574163ed0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1579400700 1 -- 172.21.15.3:0/226337128 >> 172.21.15.3:0/226337128 conn(0x7f15740af620 msgr2=0x7f15740afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1579400700 1 -- 172.21.15.3:0/226337128 shutdown_connections 2024-09-13T16:51:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.108+0000 7f1579400700 1 -- 172.21.15.3:0/226337128 wait complete. 2024-09-13T16:51:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1579400700 1 Processor -- start 2024-09-13T16:51:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1579400700 1 -- start start 2024-09-13T16:51:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1579400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f157414af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1579400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1574153450 0x7f1574149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1579400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1574153830 0x7f15741383d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1579400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1574156de0 con 0x7f15740ce5e0 2024-09-13T16:51:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1579400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157414b600 con 0x7f1574153830 2024-09-13T16:51:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1579400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157414b8e0 con 0x7f1574153450 2024-09-13T16:51:35.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f157414af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f157414af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40464/0 (socket says 172.21.15.3:40464) 2024-09-13T16:51:35.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 -- 172.21.15.3:0/414590289 learned_addr learned my addr 172.21.15.3:0/414590289 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:35.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1578a00700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1574153830 0x7f15741383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f156b400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1574153450 0x7f1574149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 -- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1574153450 msgr2=0x7f1574149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1574153450 0x7f1574149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 -- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1574153830 msgr2=0x7f15741383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1574153830 0x7f15741383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 -- 172.21.15.3:0/414590289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0094b0 con 0x7f15740ce5e0 2024-09-13T16:51:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.112+0000 7f1573e00700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f157414af90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1564001560 tx=0x7f156400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.116+0000 7f1570c00700 1 -- 172.21.15.3:0/414590289 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400acf0 con 0x7f15740ce5e0 2024-09-13T16:51:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.116+0000 7f1570c00700 1 -- 172.21.15.3:0/414590289 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156400ae70 con 0x7f15740ce5e0 2024-09-13T16:51:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.116+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157414bbc0 con 0x7f15740ce5e0 2024-09-13T16:51:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.116+0000 7f1570c00700 1 -- 172.21.15.3:0/414590289 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640072f0 con 0x7f15740ce5e0 2024-09-13T16:51:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.116+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157414c180 con 0x7f15740ce5e0 2024-09-13T16:51:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.116+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157414f710 con 0x7f15740ce5e0 2024-09-13T16:51:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.116+0000 7f1570c00700 1 -- 172.21.15.3:0/414590289 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1564007450 con 0x7f15740ce5e0 2024-09-13T16:51:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.120+0000 7f1570c00700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1550066730 0x7f1550068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.120+0000 7f1570c00700 1 -- 172.21.15.3:0/414590289 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1564084360 con 0x7f15740ce5e0 2024-09-13T16:51:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.120+0000 7f156b400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1550066730 0x7f1550068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.120+0000 7f156b400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1550066730 0x7f1550068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1574149b30 tx=0x7f155c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.128+0000 7f1570c00700 1 -- 172.21.15.3:0/414590289 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15640513a0 con 0x7f15740ce5e0 2024-09-13T16:51:35.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.316+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1574156f20 con 0x7f15740ce5e0 2024-09-13T16:51:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.316+0000 7f1570c00700 1 -- 172.21.15.3:0/414590289 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15640549c0 con 0x7f15740ce5e0 2024-09-13T16:51:35.322 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1550066730 msgr2=0x7f1550068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1550066730 0x7f1550068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1574149b30 tx=0x7f155c009040 comp rx=0 tx=0).stop 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 msgr2=0x7f157414af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f157414af90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1564001560 tx=0x7f156400a340 comp rx=0 tx=0).stop 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 shutdown_connections 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1550066730 0x7f1550068bf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1574153830 0x7f15741383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1574153450 0x7f1574149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 --2- 172.21.15.3:0/414590289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15740ce5e0 0x7f157414af90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 >> 172.21.15.3:0/414590289 conn(0x7f15740af620 msgr2=0x7f15740b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:35.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 shutdown_connections 2024-09-13T16:51:35.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.320+0000 7f1579400700 1 -- 172.21.15.3:0/414590289 wait complete. 2024-09-13T16:51:35.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:35.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.516+0000 7f46aec00700 1 Processor -- start 2024-09-13T16:51:35.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.516+0000 7f46aec00700 1 -- start start 2024-09-13T16:51:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.516+0000 7f46aec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3be0 0x7f46a8167d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.516+0000 7f46aec00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a80b1f30 con 0x7f46a8071690 2024-09-13T16:51:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.516+0000 7f46aec00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a80723e0 con 0x7f46a80b3800 2024-09-13T16:51:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.516+0000 7f46aec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80aca50 con 0x7f46a80b3be0 2024-09-13T16:51:35.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a7400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f46a8071690 0x7f46a806d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37412/0 (socket says 172.21.15.3:37412) 2024-09-13T16:51:35.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a7400700 1 -- 172.21.15.3:0/661144966 learned_addr learned my addr 172.21.15.3:0/661144966 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:35.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46acc00700 1 --2- 172.21.15.3:0/661144966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3be0 0x7f46a8167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a6a00700 1 -- 172.21.15.3:0/661144966 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943132892 0 0) 0x7f46a80b1f30 con 0x7f46a8071690 2024-09-13T16:51:35.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a6a00700 1 -- 172.21.15.3:0/661144966 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f468c0035d0 con 0x7f46a8071690 2024-09-13T16:51:35.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a6a00700 1 -- 172.21.15.3:0/661144966 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324758962 0 0) 0x7f46a80723e0 con 0x7f46a80b3800 2024-09-13T16:51:35.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a6a00700 1 -- 172.21.15.3:0/661144966 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46a80b1f30 con 0x7f46a80b3800 2024-09-13T16:51:35.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46acc00700 1 -- 172.21.15.3:0/661144966 >> v1:172.21.15.157:6789/0 conn(0x7f46a80b3800 legacy=0x7f46a8070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46acc00700 1 -- 172.21.15.3:0/661144966 >> v1:172.21.15.3:6789/0 conn(0x7f46a8071690 legacy=0x7f46a806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46acc00700 1 -- 172.21.15.3:0/661144966 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a8071b40 con 0x7f46a80b3be0 2024-09-13T16:51:35.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46acc00700 1 --2- 172.21.15.3:0/661144966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3be0 0x7f46a8167d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009dc0 tx=0x7f46a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2707c35ec1388026 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a6a00700 1 -- 172.21.15.3:0/661144966 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a000e070 con 0x7f46a80b3be0 2024-09-13T16:51:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a6a00700 1 -- 172.21.15.3:0/661144966 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a000bb20 con 0x7f46a80b3be0 2024-09-13T16:51:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46a6a00700 1 -- 172.21.15.3:0/661144966 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0012430 con 0x7f46a80b3be0 2024-09-13T16:51:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46aec00700 1 -- 172.21.15.3:0/661144966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3be0 msgr2=0x7f46a8167d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.520+0000 7f46aec00700 1 --2- 172.21.15.3:0/661144966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3be0 0x7f46a8167d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009dc0 tx=0x7f46a000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 -- 172.21.15.3:0/661144966 shutdown_connections 2024-09-13T16:51:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 --2- 172.21.15.3:0/661144966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3be0 0x7f46a8167d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 -- 172.21.15.3:0/661144966 >> 172.21.15.3:0/661144966 conn(0x7f46a806b280 msgr2=0x7f46a806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 -- 172.21.15.3:0/661144966 shutdown_connections 2024-09-13T16:51:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 -- 172.21.15.3:0/661144966 wait complete. 2024-09-13T16:51:35.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 Processor -- start 2024-09-13T16:51:35.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 -- start start 2024-09-13T16:51:35.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a8071690 0x7f46a814b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3800 0x7f46a81f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46a80b3be0 0x7f46a80cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81f6e90 con 0x7f46a80b3be0 2024-09-13T16:51:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81f68d0 con 0x7f46a8071690 2024-09-13T16:51:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.524+0000 7f46aec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81f6bb0 con 0x7f46a80b3800 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46acc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46a80b3be0 0x7f46a80cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46acc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46a80b3be0 0x7f46a80cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40478/0 (socket says 172.21.15.3:40478) 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46acc00700 1 -- 172.21.15.3:0/2735953614 learned_addr learned my addr 172.21.15.3:0/2735953614 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46a7400700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3800 0x7f46a81f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46a7e00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a8071690 0x7f46a814b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46a7400700 1 -- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a8071690 msgr2=0x7f46a814b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46a7400700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a8071690 0x7f46a814b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46a7400700 1 -- 172.21.15.3:0/2735953614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46a80b3be0 msgr2=0x7f46a80cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46a7400700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46a80b3be0 0x7f46a80cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46a7400700 1 -- 172.21.15.3:0/2735953614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a0009a70 con 0x7f46a80b3800 2024-09-13T16:51:35.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46acc00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46a80b3be0 0x7f46a80cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:35.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46a7400700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3800 0x7f46a81f5e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f46940011e0 tx=0x7f469400c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f469fe00700 1 -- 172.21.15.3:0/2735953614 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694005b90 con 0x7f46a80b3800 2024-09-13T16:51:35.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f469fe00700 1 -- 172.21.15.3:0/2735953614 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4694005cf0 con 0x7f46a80b3800 2024-09-13T16:51:35.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f469fe00700 1 -- 172.21.15.3:0/2735953614 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46940072f0 con 0x7f46a80b3800 2024-09-13T16:51:35.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a80c4040 con 0x7f46a80b3800 2024-09-13T16:51:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.528+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a814bce0 con 0x7f46a80b3800 2024-09-13T16:51:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.532+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a80c3ea0 con 0x7f46a80b3800 2024-09-13T16:51:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.532+0000 7f469fe00700 1 -- 172.21.15.3:0/2735953614 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4694007450 con 0x7f46a80b3800 2024-09-13T16:51:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.532+0000 7f469fe00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4688066730 0x7f4688068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.536+0000 7f469fe00700 1 -- 172.21.15.3:0/2735953614 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4694083b70 con 0x7f46a80b3800 2024-09-13T16:51:35.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.536+0000 7f46a7e00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4688066730 0x7f4688068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.536+0000 7f46a7e00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4688066730 0x7f4688068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f46980011e0 tx=0x7f4698009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.544+0000 7f469fe00700 1 -- 172.21.15.3:0/2735953614 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4694050b90 con 0x7f46a80b3800 2024-09-13T16:51:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.728+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46a80c4560 con 0x7f46a80b3800 2024-09-13T16:51:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f469fe00700 1 -- 172.21.15.3:0/2735953614 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4694017020 con 0x7f46a80b3800 2024-09-13T16:51:35.736 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4688066730 msgr2=0x7f4688068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4688066730 0x7f4688068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f46980011e0 tx=0x7f4698009040 comp rx=0 tx=0).stop 2024-09-13T16:51:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3800 msgr2=0x7f46a81f5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3800 0x7f46a81f5e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f46940011e0 tx=0x7f469400c700 comp rx=0 tx=0).stop 2024-09-13T16:51:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 shutdown_connections 2024-09-13T16:51:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4688066730 0x7f4688068bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46a80b3be0 0x7f46a80cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46a80b3800 0x7f46a81f5e80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 --2- 172.21.15.3:0/2735953614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a8071690 0x7f46a814b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 >> 172.21.15.3:0/2735953614 conn(0x7f46a806b280 msgr2=0x7f46a815e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 shutdown_connections 2024-09-13T16:51:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.732+0000 7f46aec00700 1 -- 172.21.15.3:0/2735953614 wait complete. 2024-09-13T16:51:35.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.940+0000 7f73dbe00700 1 Processor -- start 2024-09-13T16:51:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.940+0000 7f73dbe00700 1 -- start start 2024-09-13T16:51:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73dbe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73dc1442c0 0x7f73dc06d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73dbe00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc0b41e0 con 0x7f73dc0cded0 2024-09-13T16:51:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73dbe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc06a390 con 0x7f73dc06dcb0 2024-09-13T16:51:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73dbe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73dc06b1f0 con 0x7f73dc1442c0 2024-09-13T16:51:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73db400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f73dc0cded0 0x7f73dc15b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37414/0 (socket says 172.21.15.3:37414) 2024-09-13T16:51:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73db400700 1 -- 172.21.15.3:0/93306591 learned_addr learned my addr 172.21.15.3:0/93306591 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73daa00700 1 --2- 172.21.15.3:0/93306591 >> v2:172.21.15.157:3300/0 conn(0x7f73dc1442c0 0x7f73dc06d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73d9600700 1 -- 172.21.15.3:0/93306591 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703370135 0 0) 0x7f73dc0b41e0 con 0x7f73dc0cded0 2024-09-13T16:51:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73d9600700 1 -- 172.21.15.3:0/93306591 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73c00035d0 con 0x7f73dc0cded0 2024-09-13T16:51:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73d9600700 1 -- 172.21.15.3:0/93306591 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216370363 0 0) 0x7f73dc06a390 con 0x7f73dc06dcb0 2024-09-13T16:51:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73daa00700 1 -- 172.21.15.3:0/93306591 >> v1:172.21.15.157:6789/0 conn(0x7f73dc06dcb0 legacy=0x7f73dc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73daa00700 1 -- 172.21.15.3:0/93306591 >> v1:172.21.15.3:6789/0 conn(0x7f73dc0cded0 legacy=0x7f73dc15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73daa00700 1 -- 172.21.15.3:0/93306591 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc06a5e0 con 0x7f73dc1442c0 2024-09-13T16:51:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.944+0000 7f73daa00700 1 --2- 172.21.15.3:0/93306591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc1442c0 0x7f73dc06d770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009850 tx=0x7f73cc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79240415f28234dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73d9600700 1 -- 172.21.15.3:0/93306591 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc011830 con 0x7f73dc1442c0 2024-09-13T16:51:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73d9600700 1 -- 172.21.15.3:0/93306591 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc011990 con 0x7f73dc1442c0 2024-09-13T16:51:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73d9600700 1 -- 172.21.15.3:0/93306591 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc011d40 con 0x7f73dc1442c0 2024-09-13T16:51:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 -- 172.21.15.3:0/93306591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc1442c0 msgr2=0x7f73dc06d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 --2- 172.21.15.3:0/93306591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc1442c0 0x7f73dc06d770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009850 tx=0x7f73cc009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 -- 172.21.15.3:0/93306591 shutdown_connections 2024-09-13T16:51:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 --2- 172.21.15.3:0/93306591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc1442c0 0x7f73dc06d770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 -- 172.21.15.3:0/93306591 >> 172.21.15.3:0/93306591 conn(0x7f73dc0af620 msgr2=0x7f73dc0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 -- 172.21.15.3:0/93306591 shutdown_connections 2024-09-13T16:51:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 -- 172.21.15.3:0/93306591 wait complete. 2024-09-13T16:51:35.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 Processor -- start 2024-09-13T16:51:35.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.948+0000 7f73dbe00700 1 -- start start 2024-09-13T16:51:35.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73dbe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc06dcb0 0x7f73dc13b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73dbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc0cded0 0x7f73dc13bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73dbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73dc1442c0 0x7f73dc1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73dbe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc13cd00 con 0x7f73dc1442c0 2024-09-13T16:51:35.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73dbe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc13dd60 con 0x7f73dc0cded0 2024-09-13T16:51:35.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73dbe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc13c0f0 con 0x7f73dc06dcb0 2024-09-13T16:51:35.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73dc1442c0 0x7f73dc1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73dc1442c0 0x7f73dc1f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40480/0 (socket says 172.21.15.3:40480) 2024-09-13T16:51:35.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 -- 172.21.15.3:0/2002756146 learned_addr learned my addr 172.21.15.3:0/2002756146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:35.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73daa00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc06dcb0 0x7f73dc13b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73da000700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc0cded0 0x7f73dc13bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 -- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc06dcb0 msgr2=0x7f73dc13b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc06dcb0 0x7f73dc13b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 -- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc0cded0 msgr2=0x7f73dc13bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:35.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc0cded0 0x7f73dc13bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:35.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 -- 172.21.15.3:0/2002756146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc0094b0 con 0x7f73dc1442c0 2024-09-13T16:51:35.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73db400700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73dc1442c0 0x7f73dc1f36d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f73d400cae0 tx=0x7f73d400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73d2a00700 1 -- 172.21.15.3:0/2002756146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4007480 con 0x7f73dc1442c0 2024-09-13T16:51:35.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73d2a00700 1 -- 172.21.15.3:0/2002756146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d40075e0 con 0x7f73dc1442c0 2024-09-13T16:51:35.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73dc13c3d0 con 0x7f73dc1442c0 2024-09-13T16:51:35.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73d2a00700 1 -- 172.21.15.3:0/2002756146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4007990 con 0x7f73dc1442c0 2024-09-13T16:51:35.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.952+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc13c990 con 0x7f73dc1442c0 2024-09-13T16:51:35.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.956+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73dc13ceb0 con 0x7f73dc1442c0 2024-09-13T16:51:35.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.956+0000 7f73d2a00700 1 -- 172.21.15.3:0/2002756146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73d40113a0 con 0x7f73dc1442c0 2024-09-13T16:51:35.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.956+0000 7f73d2a00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73bc066730 0x7f73bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:35.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.960+0000 7f73d2a00700 1 -- 172.21.15.3:0/2002756146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f73d4083e00 con 0x7f73dc1442c0 2024-09-13T16:51:35.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.960+0000 7f73daa00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73bc066730 0x7f73bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:35.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.960+0000 7f73daa00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73bc066730 0x7f73bc068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0090b0 tx=0x7f73cc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:35.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:35.964+0000 7f73d2a00700 1 -- 172.21.15.3:0/2002756146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f73dc13ceb0 con 0x7f73dc1442c0 2024-09-13T16:51:36.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.144+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f73dc06d770 con 0x7f73dc1442c0 2024-09-13T16:51:36.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.144+0000 7f73d2a00700 1 -- 172.21.15.3:0/2002756146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f73d4017020 con 0x7f73dc1442c0 2024-09-13T16:51:36.149 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:36.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.144+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73bc066730 msgr2=0x7f73bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.144+0000 7f73dbe00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73bc066730 0x7f73bc068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0090b0 tx=0x7f73cc008030 comp rx=0 tx=0).stop 2024-09-13T16:51:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73dc1442c0 msgr2=0x7f73dc1f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73dc1442c0 0x7f73dc1f36d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f73d400cae0 tx=0x7f73d400cb10 comp rx=0 tx=0).stop 2024-09-13T16:51:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 shutdown_connections 2024-09-13T16:51:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73bc066730 0x7f73bc068bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73dc1442c0 0x7f73dc1f36d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc0cded0 0x7f73dc13bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 --2- 172.21.15.3:0/2002756146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc06dcb0 0x7f73dc13b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 >> 172.21.15.3:0/2002756146 conn(0x7f73dc0af620 msgr2=0x7f73dc15b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:36.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 shutdown_connections 2024-09-13T16:51:36.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.148+0000 7f73dbe00700 1 -- 172.21.15.3:0/2002756146 wait complete. 2024-09-13T16:51:36.176 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:51:36.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.344+0000 7f103de00700 1 Processor -- start 2024-09-13T16:51:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.348+0000 7f103de00700 1 -- start start 2024-09-13T16:51:36.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.348+0000 7f103de00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f103806fb50 0x7f10380cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:36.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.348+0000 7f103de00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10380cf070 con 0x7f10380b4560 2024-09-13T16:51:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.348+0000 7f103de00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10380b4be0 con 0x7f103806ff30 2024-09-13T16:51:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.348+0000 7f103de00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103815bc20 con 0x7f103806fb50 2024-09-13T16:51:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103ca00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f103806fb50 0x7f10380cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103d400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f10380b4560 0x7f103815a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37418/0 (socket says 172.21.15.3:37418) 2024-09-13T16:51:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103d400700 1 -- 172.21.15.3:0/3688440320 learned_addr learned my addr 172.21.15.3:0/3688440320 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103ca00700 1 -- 172.21.15.3:0/3688440320 >> v1:172.21.15.157:6789/0 conn(0x7f103806ff30 legacy=0x7f10380b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103ca00700 1 -- 172.21.15.3:0/3688440320 >> v1:172.21.15.3:6789/0 conn(0x7f10380b4560 legacy=0x7f103815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103ca00700 1 -- 172.21.15.3:0/3688440320 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10380cf330 con 0x7f103806fb50 2024-09-13T16:51:36.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103ca00700 1 --2- 172.21.15.3:0/3688440320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f103806fb50 0x7f10380cfe10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f102c009850 tx=0x7f102c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a7e202f931a043 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:36.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f1037400700 1 -- 172.21.15.3:0/3688440320 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c011830 con 0x7f103806fb50 2024-09-13T16:51:36.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f1037400700 1 -- 172.21.15.3:0/3688440320 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c011990 con 0x7f103806fb50 2024-09-13T16:51:36.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f1037400700 1 -- 172.21.15.3:0/3688440320 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c011d20 con 0x7f103806fb50 2024-09-13T16:51:36.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103de00700 1 -- 172.21.15.3:0/3688440320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f103806fb50 msgr2=0x7f10380cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103de00700 1 --2- 172.21.15.3:0/3688440320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f103806fb50 0x7f10380cfe10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f102c009850 tx=0x7f102c009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:36.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103de00700 1 -- 172.21.15.3:0/3688440320 shutdown_connections 2024-09-13T16:51:36.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103de00700 1 --2- 172.21.15.3:0/3688440320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f103806fb50 0x7f10380cfe10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103de00700 1 -- 172.21.15.3:0/3688440320 >> 172.21.15.3:0/3688440320 conn(0x7f10380b2910 msgr2=0x7f10380b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:36.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103de00700 1 -- 172.21.15.3:0/3688440320 shutdown_connections 2024-09-13T16:51:36.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.352+0000 7f103de00700 1 -- 172.21.15.3:0/3688440320 wait complete. 2024-09-13T16:51:36.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103de00700 1 Processor -- start 2024-09-13T16:51:36.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103de00700 1 -- start start 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103de00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f103806fb50 0x7f103814ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103806ff30 0x7f103814ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10380b4560 0x7f10381f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103de00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1038151260 con 0x7f10380b4560 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103de00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103814d2a0 con 0x7f103806ff30 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103de00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1038143540 con 0x7f103806fb50 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10380b4560 0x7f10381f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10380b4560 0x7f10381f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40500/0 (socket says 172.21.15.3:40500) 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103d400700 1 -- 172.21.15.3:0/2250038246 learned_addr learned my addr 172.21.15.3:0/2250038246 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f103ca00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f103806fb50 0x7f103814ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.356+0000 7f1037e00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103806ff30 0x7f103814ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103d400700 1 -- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f103806fb50 msgr2=0x7f103814ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103d400700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f103806fb50 0x7f103814ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103d400700 1 -- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103806ff30 msgr2=0x7f103814ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103d400700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103806ff30 0x7f103814ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103d400700 1 -- 172.21.15.3:0/2250038246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c0094b0 con 0x7f10380b4560 2024-09-13T16:51:36.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103d400700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10380b4560 0x7f10381f3fb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f103000ca80 tx=0x7f103000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:36.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f1034c00700 1 -- 172.21.15.3:0/2250038246 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030007480 con 0x7f10380b4560 2024-09-13T16:51:36.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f1034c00700 1 -- 172.21.15.3:0/2250038246 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10300075e0 con 0x7f10380b4560 2024-09-13T16:51:36.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f1034c00700 1 -- 172.21.15.3:0/2250038246 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030007950 con 0x7f10380b4560 2024-09-13T16:51:36.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1038143820 con 0x7f10380b4560 2024-09-13T16:51:36.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1038143de0 con 0x7f10380b4560 2024-09-13T16:51:36.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.360+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10380cfe10 con 0x7f10380b4560 2024-09-13T16:51:36.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.364+0000 7f1034c00700 1 -- 172.21.15.3:0/2250038246 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f103001b3b0 con 0x7f10380b4560 2024-09-13T16:51:36.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.364+0000 7f1034c00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1018066730 0x7f1018068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:36.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.364+0000 7f1034c00700 1 -- 172.21.15.3:0/2250038246 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1030083d50 con 0x7f10380b4560 2024-09-13T16:51:36.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.364+0000 7f103ca00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1018066730 0x7f1018068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:36.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.364+0000 7f103ca00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1018066730 0x7f1018068bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f102c009fd0 tx=0x7f102c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:36.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.372+0000 7f1034c00700 1 -- 172.21.15.3:0/2250038246 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1030050c80 con 0x7f10380b4560 2024-09-13T16:51:36.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.568+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f10380b5920 con 0x7f10380b4560 2024-09-13T16:51:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.568+0000 7f1034c00700 1 -- 172.21.15.3:0/2250038246 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1030017070 con 0x7f10380b4560 2024-09-13T16:51:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1018066730 msgr2=0x7f1018068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1018066730 0x7f1018068bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f102c009fd0 tx=0x7f102c008030 comp rx=0 tx=0).stop 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10380b4560 msgr2=0x7f10381f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10380b4560 0x7f10381f3fb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f103000ca80 tx=0x7f103000cab0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 shutdown_connections 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1018066730 0x7f1018068bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10380b4560 0x7f10381f3fb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103806ff30 0x7f103814ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 --2- 172.21.15.3:0/2250038246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f103806fb50 0x7f103814ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 >> 172.21.15.3:0/2250038246 conn(0x7f10380b2910 msgr2=0x7f10380b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 shutdown_connections 2024-09-13T16:51:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.572+0000 7f103de00700 1 -- 172.21.15.3:0/2250038246 wait complete. 2024-09-13T16:51:36.581 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-13T16:51:36.602 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T16:51:36.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:36.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-13T16:51:36.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:51:36.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:36.657 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T16:51:36.668 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:51:36.669 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:36.687 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:51:36.687 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:36.750 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:51:36.750 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:36.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:36.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.984+0000 7f6e75600700 1 Processor -- start 2024-09-13T16:51:36.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.984+0000 7f6e75600700 1 -- start start 2024-09-13T16:51:36.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e75600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cfa0 0x7f6e700ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:36.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e75600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e700ae1e0 0x7f6e700cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:36.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e75600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7006a390 con 0x7f6e7015cbc0 2024-09-13T16:51:36.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e75600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7006a670 con 0x7f6e700ae1e0 2024-09-13T16:51:36.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e75600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7006a950 con 0x7f6e7015cfa0 2024-09-13T16:51:36.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e74c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e700ae1e0 0x7f6e700cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:36.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e67400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cfa0 0x7f6e700ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:36.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e74c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e700ae1e0 0x7f6e700cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:57682/0 (socket says 172.21.15.3:57682) 2024-09-13T16:51:36.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e74c00700 1 -- 172.21.15.3:0/3297923895 learned_addr learned my addr 172.21.15.3:0/3297923895 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:36.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e74c00700 1 -- 172.21.15.3:0/3297923895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cfa0 msgr2=0x7f6e700ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:36.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e74c00700 1 --2- 172.21.15.3:0/3297923895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cfa0 0x7f6e700ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e74c00700 1 -- 172.21.15.3:0/3297923895 >> v1:172.21.15.157:6789/0 conn(0x7f6e7015cbc0 legacy=0x7f6e700adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e74c00700 1 -- 172.21.15.3:0/3297923895 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7006ac30 con 0x7f6e700ae1e0 2024-09-13T16:51:36.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e74c00700 1 --2- 172.21.15.3:0/3297923895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e700ae1e0 0x7f6e700cd0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009910 tx=0x7f6e68009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ef2ca45afd8acd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:36.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e6f400700 1 -- 172.21.15.3:0/3297923895 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68011830 con 0x7f6e700ae1e0 2024-09-13T16:51:36.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e6f400700 1 -- 172.21.15.3:0/3297923895 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68011990 con 0x7f6e700ae1e0 2024-09-13T16:51:36.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.988+0000 7f6e6f400700 1 -- 172.21.15.3:0/3297923895 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68011d40 con 0x7f6e700ae1e0 2024-09-13T16:51:36.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.992+0000 7f6e75600700 1 -- 172.21.15.3:0/3297923895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e700ae1e0 msgr2=0x7f6e700cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:36.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.992+0000 7f6e75600700 1 --2- 172.21.15.3:0/3297923895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e700ae1e0 0x7f6e700cd0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009910 tx=0x7f6e68009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:36.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.992+0000 7f6e75600700 1 -- 172.21.15.3:0/3297923895 shutdown_connections 2024-09-13T16:51:36.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.992+0000 7f6e75600700 1 --2- 172.21.15.3:0/3297923895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e700ae1e0 0x7f6e700cd0a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.992+0000 7f6e75600700 1 --2- 172.21.15.3:0/3297923895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cfa0 0x7f6e700ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:36.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.992+0000 7f6e75600700 1 -- 172.21.15.3:0/3297923895 >> 172.21.15.3:0/3297923895 conn(0x7f6e700b0cf0 msgr2=0x7f6e700b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:36.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.992+0000 7f6e75600700 1 -- 172.21.15.3:0/3297923895 shutdown_connections 2024-09-13T16:51:36.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.992+0000 7f6e75600700 1 -- 172.21.15.3:0/3297923895 wait complete. 2024-09-13T16:51:36.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e75600700 1 Processor -- start 2024-09-13T16:51:36.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e75600700 1 -- start start 2024-09-13T16:51:37.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e75600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e700ae1e0 0x7f6e701f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e75600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cbc0 0x7f6e701f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e75600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7015cfa0 0x7f6e700af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e75600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e701f2b90 con 0x7f6e700ae1e0 2024-09-13T16:51:37.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e75600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e701f2e70 con 0x7f6e7015cfa0 2024-09-13T16:51:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e75600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e701f3150 con 0x7f6e7015cbc0 2024-09-13T16:51:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e700ae1e0 0x7f6e701f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e700ae1e0 0x7f6e701f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40508/0 (socket says 172.21.15.3:40508) 2024-09-13T16:51:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 -- 172.21.15.3:0/3743541340 learned_addr learned my addr 172.21.15.3:0/3743541340 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e74c00700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7015cfa0 0x7f6e700af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e67400700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cbc0 0x7f6e701f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 -- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cbc0 msgr2=0x7f6e701f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cbc0 0x7f6e701f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 -- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7015cfa0 msgr2=0x7f6e700af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7015cfa0 0x7f6e700af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 -- 172.21.15.3:0/3743541340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009570 con 0x7f6e700ae1e0 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e67400700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cbc0 0x7f6e701f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:36.996+0000 7f6e6fe00700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e700ae1e0 0x7f6e701f3a40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e60001ed0 tx=0x7f6e600021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.000+0000 7f6e6cc00700 1 -- 172.21.15.3:0/3743541340 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60003c20 con 0x7f6e700ae1e0 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.000+0000 7f6e6cc00700 1 -- 172.21.15.3:0/3743541340 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60003d80 con 0x7f6e700ae1e0 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.000+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e701f3430 con 0x7f6e700ae1e0 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.000+0000 7f6e6cc00700 1 -- 172.21.15.3:0/3743541340 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000abf0 con 0x7f6e700ae1e0 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.000+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e701f5bb0 con 0x7f6e700ae1e0 2024-09-13T16:51:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.000+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e700adad0 con 0x7f6e700ae1e0 2024-09-13T16:51:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.004+0000 7f6e6cc00700 1 -- 172.21.15.3:0/3743541340 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e6000ad50 con 0x7f6e700ae1e0 2024-09-13T16:51:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.004+0000 7f6e6cc00700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6e4c066730 0x7f6e4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.004+0000 7f6e6cc00700 1 -- 172.21.15.3:0/3743541340 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6e600846e0 con 0x7f6e700ae1e0 2024-09-13T16:51:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.004+0000 7f6e67400700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6e4c066730 0x7f6e4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.004+0000 7f6e67400700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6e4c066730 0x7f6e4c068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6e701f0470 tx=0x7f6e58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.012+0000 7f6e6cc00700 1 -- 172.21.15.3:0/3743541340 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6e600515f0 con 0x7f6e700ae1e0 2024-09-13T16:51:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.208+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e7006cbc0 con 0x7f6e700ae1e0 2024-09-13T16:51:37.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.208+0000 7f6e6cc00700 1 -- 172.21.15.3:0/3743541340 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f6e60054c10 con 0x7f6e700ae1e0 2024-09-13T16:51:37.215 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:37.216 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6e4c066730 msgr2=0x7f6e4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6e4c066730 0x7f6e4c068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6e701f0470 tx=0x7f6e58008040 comp rx=0 tx=0).stop 2024-09-13T16:51:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e700ae1e0 msgr2=0x7f6e701f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e700ae1e0 0x7f6e701f3a40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e60001ed0 tx=0x7f6e600021e0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 shutdown_connections 2024-09-13T16:51:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6e4c066730 0x7f6e4c068bf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7015cfa0 0x7f6e700af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7015cbc0 0x7f6e701f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 --2- 172.21.15.3:0/3743541340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e700ae1e0 0x7f6e701f3a40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 >> 172.21.15.3:0/3743541340 conn(0x7f6e700b0cf0 msgr2=0x7f6e700cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 shutdown_connections 2024-09-13T16:51:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.216+0000 7f6e75600700 1 -- 172.21.15.3:0/3743541340 wait complete. 2024-09-13T16:51:37.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:37.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:37.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.416+0000 7fea32a00700 1 Processor -- start 2024-09-13T16:51:37.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.416+0000 7fea32a00700 1 -- start start 2024-09-13T16:51:37.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.416+0000 7fea32a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea2c06f970 0x7fea2c06fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.416+0000 7fea32a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c0ce620 con 0x7fea2c15ccc0 2024-09-13T16:51:37.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.416+0000 7fea32a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c0ce900 con 0x7fea2c0cae70 2024-09-13T16:51:37.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.416+0000 7fea32a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c070320 con 0x7fea2c06f970 2024-09-13T16:51:37.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2be00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea2c15ccc0 0x7fea2c0cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37428/0 (socket says 172.21.15.3:37428) 2024-09-13T16:51:37.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2be00700 1 -- 172.21.15.3:0/1075306536 learned_addr learned my addr 172.21.15.3:0/1075306536 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:37.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2b400700 1 --2- 172.21.15.3:0/1075306536 >> v2:172.21.15.157:3300/0 conn(0x7fea2c06f970 0x7fea2c06fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2aa00700 1 -- 172.21.15.3:0/1075306536 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208069994 0 0) 0x7fea2c0ce620 con 0x7fea2c15ccc0 2024-09-13T16:51:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2aa00700 1 -- 172.21.15.3:0/1075306536 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea100035d0 con 0x7fea2c15ccc0 2024-09-13T16:51:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2b400700 1 -- 172.21.15.3:0/1075306536 >> v1:172.21.15.157:6789/0 conn(0x7fea2c0cae70 legacy=0x7fea2c0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2b400700 1 -- 172.21.15.3:0/1075306536 >> v1:172.21.15.3:6789/0 conn(0x7fea2c15ccc0 legacy=0x7fea2c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2b400700 1 -- 172.21.15.3:0/1075306536 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0cebe0 con 0x7fea2c06f970 2024-09-13T16:51:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2b400700 1 --2- 172.21.15.3:0/1075306536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c06f970 0x7fea2c06fde0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea18009850 tx=0x7fea18009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f9996e1dab6614 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:37.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2aa00700 1 -- 172.21.15.3:0/1075306536 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18011830 con 0x7fea2c06f970 2024-09-13T16:51:37.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2aa00700 1 -- 172.21.15.3:0/1075306536 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18011990 con 0x7fea2c06f970 2024-09-13T16:51:37.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea2aa00700 1 -- 172.21.15.3:0/1075306536 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18011d40 con 0x7fea2c06f970 2024-09-13T16:51:37.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea32a00700 1 -- 172.21.15.3:0/1075306536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c06f970 msgr2=0x7fea2c06fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea32a00700 1 --2- 172.21.15.3:0/1075306536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c06f970 0x7fea2c06fde0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea18009850 tx=0x7fea18009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea32a00700 1 -- 172.21.15.3:0/1075306536 shutdown_connections 2024-09-13T16:51:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea32a00700 1 --2- 172.21.15.3:0/1075306536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c06f970 0x7fea2c06fde0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.420+0000 7fea32a00700 1 -- 172.21.15.3:0/1075306536 >> 172.21.15.3:0/1075306536 conn(0x7fea2c0b2b00 msgr2=0x7fea2c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 -- 172.21.15.3:0/1075306536 shutdown_connections 2024-09-13T16:51:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 -- 172.21.15.3:0/1075306536 wait complete. 2024-09-13T16:51:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 Processor -- start 2024-09-13T16:51:37.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 -- start start 2024-09-13T16:51:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2c06f970 0x7fea2c1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c0cae70 0x7fea2c147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2c15ccc0 0x7fea2c152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c1517f0 con 0x7fea2c06f970 2024-09-13T16:51:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c160310 con 0x7fea2c0cae70 2024-09-13T16:51:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea32a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c1605f0 con 0x7fea2c15ccc0 2024-09-13T16:51:37.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.424+0000 7fea2be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2c06f970 0x7fea2c1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2c06f970 0x7fea2c1512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40512/0 (socket says 172.21.15.3:40512) 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2be00700 1 -- 172.21.15.3:0/3800693673 learned_addr learned my addr 172.21.15.3:0/3800693673 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2b400700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c0cae70 0x7fea2c147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea30a00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2c15ccc0 0x7fea2c152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2be00700 1 -- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2c15ccc0 msgr2=0x7fea2c152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2be00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2c15ccc0 0x7fea2c152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2be00700 1 -- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c0cae70 msgr2=0x7fea2c147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2be00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c0cae70 0x7fea2c147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2be00700 1 -- 172.21.15.3:0/3800693673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea180094b0 con 0x7fea2c06f970 2024-09-13T16:51:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea2be00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2c06f970 0x7fea2c1512b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00c4d0 tx=0x7fea1c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:37.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea23e00700 1 -- 172.21.15.3:0/3800693673 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c005d90 con 0x7fea2c06f970 2024-09-13T16:51:37.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea23e00700 1 -- 172.21.15.3:0/3800693673 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c0070d0 con 0x7fea2c06f970 2024-09-13T16:51:37.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c153b20 con 0x7fea2c06f970 2024-09-13T16:51:37.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea23e00700 1 -- 172.21.15.3:0/3800693673 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c007480 con 0x7fea2c06f970 2024-09-13T16:51:37.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c15ff30 con 0x7fea2c06f970 2024-09-13T16:51:37.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.428+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2c0ce1f0 con 0x7fea2c06f970 2024-09-13T16:51:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.432+0000 7fea23e00700 1 -- 172.21.15.3:0/3800693673 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea1c0075e0 con 0x7fea2c06f970 2024-09-13T16:51:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.432+0000 7fea23e00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea0c066760 0x7fea0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.432+0000 7fea23e00700 1 -- 172.21.15.3:0/3800693673 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fea1c083ae0 con 0x7fea2c06f970 2024-09-13T16:51:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.436+0000 7fea2b400700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea0c066760 0x7fea0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.436+0000 7fea2b400700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea0c066760 0x7fea0c068c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea180090b0 tx=0x7fea18008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.440+0000 7fea23e00700 1 -- 172.21.15.3:0/3800693673 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fea1c0509f0 con 0x7fea2c06f970 2024-09-13T16:51:37.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.620+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea2c0d1550 con 0x7fea2c06f970 2024-09-13T16:51:37.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.624+0000 7fea23e00700 1 -- 172.21.15.3:0/3800693673 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea1c054010 con 0x7fea2c06f970 2024-09-13T16:51:37.628 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:37.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.624+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea0c066760 msgr2=0x7fea0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea0c066760 0x7fea0c068c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea180090b0 tx=0x7fea18008030 comp rx=0 tx=0).stop 2024-09-13T16:51:37.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2c06f970 msgr2=0x7fea2c1512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2c06f970 0x7fea2c1512b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00c4d0 tx=0x7fea1c00c890 comp rx=0 tx=0).stop 2024-09-13T16:51:37.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 shutdown_connections 2024-09-13T16:51:37.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea0c066760 0x7fea0c068c20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2c15ccc0 0x7fea2c152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2c0cae70 0x7fea2c147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 --2- 172.21.15.3:0/3800693673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2c06f970 0x7fea2c1512b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 >> 172.21.15.3:0/3800693673 conn(0x7fea2c0b2b00 msgr2=0x7fea2c0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:37.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 shutdown_connections 2024-09-13T16:51:37.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.628+0000 7fea32a00700 1 -- 172.21.15.3:0/3800693673 wait complete. 2024-09-13T16:51:37.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:37.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.824+0000 7fe50d400700 1 Processor -- start 2024-09-13T16:51:37.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.824+0000 7fe50d400700 1 -- start start 2024-09-13T16:51:37.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.828+0000 7fe50d400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5080b45a0 0x7fe50816bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.828+0000 7fe50d400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5080cc710 con 0x7fe50806af80 2024-09-13T16:51:37.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.828+0000 7fe50d400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5080cc9f0 con 0x7fe508161d20 2024-09-13T16:51:37.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.828+0000 7fe50d400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5080cd390 con 0x7fe5080b45a0 2024-09-13T16:51:37.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.828+0000 7fe507e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe50806af80 0x7fe50806b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37442/0 (socket says 172.21.15.3:37442) 2024-09-13T16:51:37.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.828+0000 7fe507e00700 1 -- 172.21.15.3:0/2846593498 learned_addr learned my addr 172.21.15.3:0/2846593498 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:37.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.828+0000 7fe50ca00700 1 --2- 172.21.15.3:0/2846593498 >> v2:172.21.15.3:3300/0 conn(0x7fe5080b45a0 0x7fe50816bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50ca00700 1 -- 172.21.15.3:0/2846593498 >> v1:172.21.15.157:6789/0 conn(0x7fe508161d20 legacy=0x7fe5080b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50ca00700 1 -- 172.21.15.3:0/2846593498 >> v1:172.21.15.3:6789/0 conn(0x7fe50806af80 legacy=0x7fe50806b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50ca00700 1 -- 172.21.15.3:0/2846593498 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080cccd0 con 0x7fe5080b45a0 2024-09-13T16:51:37.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe506a00700 1 -- 172.21.15.3:0/2846593498 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990542504 0 0) 0x7fe5080cc710 con 0x7fe50806af80 2024-09-13T16:51:37.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50ca00700 1 --2- 172.21.15.3:0/2846593498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 0x7fe50816bb60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe500009d90 tx=0x7fe50000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900f7afc106ae028 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:37.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe506a00700 1 -- 172.21.15.3:0/2846593498 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50000e070 con 0x7fe5080b45a0 2024-09-13T16:51:37.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe506a00700 1 -- 172.21.15.3:0/2846593498 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50000bbd0 con 0x7fe5080b45a0 2024-09-13T16:51:37.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe506a00700 1 -- 172.21.15.3:0/2846593498 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500012430 con 0x7fe5080b45a0 2024-09-13T16:51:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50d400700 1 -- 172.21.15.3:0/2846593498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 msgr2=0x7fe50816bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50d400700 1 --2- 172.21.15.3:0/2846593498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 0x7fe50816bb60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe500009d90 tx=0x7fe50000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50d400700 1 -- 172.21.15.3:0/2846593498 shutdown_connections 2024-09-13T16:51:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50d400700 1 --2- 172.21.15.3:0/2846593498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 0x7fe50816bb60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50d400700 1 -- 172.21.15.3:0/2846593498 >> 172.21.15.3:0/2846593498 conn(0x7fe5080710b0 msgr2=0x7fe5080714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50d400700 1 -- 172.21.15.3:0/2846593498 shutdown_connections 2024-09-13T16:51:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.832+0000 7fe50d400700 1 -- 172.21.15.3:0/2846593498 wait complete. 2024-09-13T16:51:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50d400700 1 Processor -- start 2024-09-13T16:51:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50d400700 1 -- start start 2024-09-13T16:51:37.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806af80 0x7fe508148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 0x7fe50815fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50d400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe508161d20 0x7fe508160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50d400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe508149360 con 0x7fe5080b45a0 2024-09-13T16:51:37.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50d400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe508160950 con 0x7fe50806af80 2024-09-13T16:51:37.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50d400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe508160c30 con 0x7fe508161d20 2024-09-13T16:51:37.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe507e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806af80 0x7fe508148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe507400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 0x7fe50815fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe507e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806af80 0x7fe508148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:57718/0 (socket says 172.21.15.3:57718) 2024-09-13T16:51:37.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe507e00700 1 -- 172.21.15.3:0/2653286042 learned_addr learned my addr 172.21.15.3:0/2653286042 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:37.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.836+0000 7fe50ca00700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe508161d20 0x7fe508160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe507400700 1 -- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe508161d20 msgr2=0x7fe508160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe507400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe508161d20 0x7fe508160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe507400700 1 -- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806af80 msgr2=0x7fe508148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe507400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806af80 0x7fe508148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe507400700 1 -- 172.21.15.3:0/2653286042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe500009a70 con 0x7fe5080b45a0 2024-09-13T16:51:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe507e00700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806af80 0x7fe508148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe507400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 0x7fe50815fdd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c410 tx=0x7fe4f4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe4ffe00700 1 -- 172.21.15.3:0/2653286042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40052d0 con 0x7fe5080b45a0 2024-09-13T16:51:37.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe4ffe00700 1 -- 172.21.15.3:0/2653286042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f4005430 con 0x7fe5080b45a0 2024-09-13T16:51:37.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe508160f10 con 0x7fe5080b45a0 2024-09-13T16:51:37.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe4ffe00700 1 -- 172.21.15.3:0/2653286042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40055a0 con 0x7fe5080b45a0 2024-09-13T16:51:37.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5081614d0 con 0x7fe5080b45a0 2024-09-13T16:51:37.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.840+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5080cc710 con 0x7fe5080b45a0 2024-09-13T16:51:37.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.844+0000 7fe4ffe00700 1 -- 172.21.15.3:0/2653286042 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4f40038e0 con 0x7fe5080b45a0 2024-09-13T16:51:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.844+0000 7fe4ffe00700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4e8066730 0x7fe4e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.844+0000 7fe4ffe00700 1 -- 172.21.15.3:0/2653286042 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe4f4083820 con 0x7fe5080b45a0 2024-09-13T16:51:37.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.844+0000 7fe507e00700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4e8066730 0x7fe4e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.844+0000 7fe507e00700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4e8066730 0x7fe4e8068bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8000f60 tx=0x7fe4f8001960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:37.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:37.852+0000 7fe4ffe00700 1 -- 172.21.15.3:0/2653286042 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe4f4050740 con 0x7fe5080b45a0 2024-09-13T16:51:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.036+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe50806a4c0 con 0x7fe5080b45a0 2024-09-13T16:51:38.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.036+0000 7fe4ffe00700 1 -- 172.21.15.3:0/2653286042 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4f4053d60 con 0x7fe5080b45a0 2024-09-13T16:51:38.042 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4e8066730 msgr2=0x7fe4e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4e8066730 0x7fe4e8068bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8000f60 tx=0x7fe4f8001960 comp rx=0 tx=0).stop 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 msgr2=0x7fe50815fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 0x7fe50815fdd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c410 tx=0x7fe4f4001e70 comp rx=0 tx=0).stop 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 shutdown_connections 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe4e8066730 0x7fe4e8068bf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe508161d20 0x7fe508160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5080b45a0 0x7fe50815fdd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 --2- 172.21.15.3:0/2653286042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806af80 0x7fe508148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 >> 172.21.15.3:0/2653286042 conn(0x7fe5080710b0 msgr2=0x7fe5080b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.040+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 shutdown_connections 2024-09-13T16:51:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.044+0000 7fe50d400700 1 -- 172.21.15.3:0/2653286042 wait complete. 2024-09-13T16:51:38.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:38.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.248+0000 7f2bfb200700 1 Processor -- start 2024-09-13T16:51:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.248+0000 7f2bfb200700 1 -- start start 2024-09-13T16:51:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.248+0000 7f2bfb200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2bfc1444f0 0x7f2bfc0d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.248+0000 7f2bfb200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0d0680 0x7f2bfc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.248+0000 7f2bfb200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2bfc0b5570 0x7f2bfc15ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.248+0000 7f2bfb200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b5a30 con 0x7f2bfc0b5570 2024-09-13T16:51:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.248+0000 7f2bfb200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b5d10 con 0x7f2bfc1444f0 2024-09-13T16:51:38.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.248+0000 7f2bfb200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0cd5b0 con 0x7f2bfc0d0680 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2bfc1444f0 0x7f2bfc0d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfa800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2bfc0b5570 0x7f2bfc15ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2bfc1444f0 0x7f2bfc0d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:57728/0 (socket says 172.21.15.3:57728) 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 -- 172.21.15.3:0/348683688 learned_addr learned my addr 172.21.15.3:0/348683688 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9400700 1 --2- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0d0680 0x7f2bfc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 -- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0d0680 msgr2=0x7f2bfc15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 --2- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0d0680 0x7f2bfc15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 -- 172.21.15.3:0/348683688 >> v2:172.21.15.3:3300/0 conn(0x7f2bfc0b5570 msgr2=0x7f2bfc15ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 --2- 172.21.15.3:0/348683688 >> v2:172.21.15.3:3300/0 conn(0x7f2bfc0b5570 0x7f2bfc15ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 -- 172.21.15.3:0/348683688 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0cd890 con 0x7f2bfc1444f0 2024-09-13T16:51:38.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9400700 1 --2- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0d0680 0x7f2bfc15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf9e00700 1 --2- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 0x7f2bfc0d0140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0b2650 tx=0x7f2bec009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=992ed9b340f4310f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf8a00700 1 -- 172.21.15.3:0/348683688 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec011830 con 0x7f2bfc1444f0 2024-09-13T16:51:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf8a00700 1 -- 172.21.15.3:0/348683688 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec011990 con 0x7f2bfc1444f0 2024-09-13T16:51:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bf8a00700 1 -- 172.21.15.3:0/348683688 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec011d20 con 0x7f2bfc1444f0 2024-09-13T16:51:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 -- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 msgr2=0x7f2bfc0d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 --2- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 0x7f2bfc0d0140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0b2650 tx=0x7f2bec009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 -- 172.21.15.3:0/348683688 shutdown_connections 2024-09-13T16:51:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 --2- 172.21.15.3:0/348683688 >> v2:172.21.15.3:3300/0 conn(0x7f2bfc0b5570 0x7f2bfc15ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 --2- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0d0680 0x7f2bfc15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 --2- 172.21.15.3:0/348683688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 0x7f2bfc0d0140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 -- 172.21.15.3:0/348683688 >> 172.21.15.3:0/348683688 conn(0x7f2bfc0721a0 msgr2=0x7f2bfc06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 -- 172.21.15.3:0/348683688 shutdown_connections 2024-09-13T16:51:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.252+0000 7f2bfb200700 1 -- 172.21.15.3:0/348683688 wait complete. 2024-09-13T16:51:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.256+0000 7f2bfb200700 1 Processor -- start 2024-09-13T16:51:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.256+0000 7f2bfb200700 1 -- start start 2024-09-13T16:51:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.256+0000 7f2bfb200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0b5570 0x7f2bfc1efa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.256+0000 7f2bfb200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bfc0d0680 0x7f2bfc1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.256+0000 7f2bfb200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 0x7f2bfc159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bfb200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc1effd0 con 0x7f2bfc0d0680 2024-09-13T16:51:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bfb200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc140180 con 0x7f2bfc1444f0 2024-09-13T16:51:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bfb200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc15a540 con 0x7f2bfc0b5570 2024-09-13T16:51:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bfc0d0680 0x7f2bfc1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bfc0d0680 0x7f2bfc1599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40570/0 (socket says 172.21.15.3:40570) 2024-09-13T16:51:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 -- 172.21.15.3:0/1941715633 learned_addr learned my addr 172.21.15.3:0/1941715633 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9e00700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0b5570 0x7f2bfc1efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bfa800700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 0x7f2bfc159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 -- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0b5570 msgr2=0x7f2bfc1efa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0b5570 0x7f2bfc1efa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 -- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 msgr2=0x7f2bfc159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 0x7f2bfc159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 -- 172.21.15.3:0/1941715633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec009570 con 0x7f2bfc0d0680 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bfa800700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 0x7f2bfc159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf9400700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bfc0d0680 0x7f2bfc1599d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2be800e730 tx=0x7f2be800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf2000700 1 -- 172.21.15.3:0/1941715633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8004230 con 0x7f2bfc0d0680 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf2000700 1 -- 172.21.15.3:0/1941715633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be8004390 con 0x7f2bfc0d0680 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bf2000700 1 -- 172.21.15.3:0/1941715633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80036c0 con 0x7f2bfc0d0680 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc15a820 con 0x7f2bfc0d0680 2024-09-13T16:51:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc1f2810 con 0x7f2bfc0d0680 2024-09-13T16:51:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.260+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc15ad40 con 0x7f2bfc0d0680 2024-09-13T16:51:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.264+0000 7f2bf2000700 1 -- 172.21.15.3:0/1941715633 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2be8003840 con 0x7f2bfc0d0680 2024-09-13T16:51:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.264+0000 7f2bf2000700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2bdc066730 0x7f2bdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.264+0000 7f2bf2000700 1 -- 172.21.15.3:0/1941715633 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2be8084030 con 0x7f2bfc0d0680 2024-09-13T16:51:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.264+0000 7f2bf9e00700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2bdc066730 0x7f2bdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.264+0000 7f2bf9e00700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2bdc066730 0x7f2bdc068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2bec000c00 tx=0x7f2bec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:38.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.272+0000 7f2bf2000700 1 -- 172.21.15.3:0/1941715633 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2bfc15ad40 con 0x7f2bfc0d0680 2024-09-13T16:51:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.452+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bfc13ff50 con 0x7f2bfc0d0680 2024-09-13T16:51:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bf2000700 1 -- 172.21.15.3:0/1941715633 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bfc13ff50 con 0x7f2bfc0d0680 2024-09-13T16:51:38.460 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2bdc066730 msgr2=0x7f2bdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2bdc066730 0x7f2bdc068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2bec000c00 tx=0x7f2bec008030 comp rx=0 tx=0).stop 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bfc0d0680 msgr2=0x7f2bfc1599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bfc0d0680 0x7f2bfc1599d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2be800e730 tx=0x7f2be800eaf0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 shutdown_connections 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2bdc066730 0x7f2bdc068bf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bfc1444f0 0x7f2bfc159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bfc0d0680 0x7f2bfc1599d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 --2- 172.21.15.3:0/1941715633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bfc0b5570 0x7f2bfc1efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 >> 172.21.15.3:0/1941715633 conn(0x7f2bfc0721a0 msgr2=0x7f2bfc15e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 shutdown_connections 2024-09-13T16:51:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.456+0000 7f2bfb200700 1 -- 172.21.15.3:0/1941715633 wait complete. 2024-09-13T16:51:38.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:38.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.660+0000 7f54fd800700 1 Processor -- start 2024-09-13T16:51:38.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.660+0000 7f54fd800700 1 -- start start 2024-09-13T16:51:38.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fd800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54f815e7b0 0x7f54f815af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fd800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54f815eb90 0x7f54f815f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fd800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f81407b0 0x7f54f8140c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fd800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f815b490 con 0x7f54f815e7b0 2024-09-13T16:51:38.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fd800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f8140320 con 0x7f54f815eb90 2024-09-13T16:51:38.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fd800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f815b6c0 con 0x7f54f81407b0 2024-09-13T16:51:38.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54f815e7b0 0x7f54f815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54f815e7b0 0x7f54f815af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40572/0 (socket says 172.21.15.3:40572) 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 -- 172.21.15.3:0/1331329732 learned_addr learned my addr 172.21.15.3:0/1331329732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fce00700 1 --2- 172.21.15.3:0/1331329732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f81407b0 0x7f54f8140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7400700 1 --2- 172.21.15.3:0/1331329732 >> v2:172.21.15.157:3300/0 conn(0x7f54f815eb90 0x7f54f815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 -- 172.21.15.3:0/1331329732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f81407b0 msgr2=0x7f54f8140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 --2- 172.21.15.3:0/1331329732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f81407b0 0x7f54f8140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 -- 172.21.15.3:0/1331329732 >> v2:172.21.15.157:3300/0 conn(0x7f54f815eb90 msgr2=0x7f54f815f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 --2- 172.21.15.3:0/1331329732 >> v2:172.21.15.157:3300/0 conn(0x7f54f815eb90 0x7f54f815f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 -- 172.21.15.3:0/1331329732 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f815b9a0 con 0x7f54f815e7b0 2024-09-13T16:51:38.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7400700 1 --2- 172.21.15.3:0/1331329732 >> v2:172.21.15.157:3300/0 conn(0x7f54f815eb90 0x7f54f815f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:38.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f7e00700 1 --2- 172.21.15.3:0/1331329732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f815af50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f54f806a4f0 tx=0x7f54e8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2233117a84b5634a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:38.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f6a00700 1 -- 172.21.15.3:0/1331329732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8011830 con 0x7f54f815e7b0 2024-09-13T16:51:38.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f6a00700 1 -- 172.21.15.3:0/1331329732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e8011990 con 0x7f54f815e7b0 2024-09-13T16:51:38.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54f6a00700 1 -- 172.21.15.3:0/1331329732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8011d20 con 0x7f54f815e7b0 2024-09-13T16:51:38.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fd800700 1 -- 172.21.15.3:0/1331329732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 msgr2=0x7f54f815af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.664+0000 7f54fd800700 1 --2- 172.21.15.3:0/1331329732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f815af50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f54f806a4f0 tx=0x7f54e8009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 -- 172.21.15.3:0/1331329732 shutdown_connections 2024-09-13T16:51:38.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 --2- 172.21.15.3:0/1331329732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f81407b0 0x7f54f8140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 --2- 172.21.15.3:0/1331329732 >> v2:172.21.15.157:3300/0 conn(0x7f54f815eb90 0x7f54f815f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 --2- 172.21.15.3:0/1331329732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f815af50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 -- 172.21.15.3:0/1331329732 >> 172.21.15.3:0/1331329732 conn(0x7f54f80712c0 msgr2=0x7f54f80716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:38.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 -- 172.21.15.3:0/1331329732 shutdown_connections 2024-09-13T16:51:38.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 -- 172.21.15.3:0/1331329732 wait complete. 2024-09-13T16:51:38.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 Processor -- start 2024-09-13T16:51:38.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 -- start start 2024-09-13T16:51:38.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f81407b0 0x7f54f813d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f8143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f815eb90 0x7f54f81439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f806c880 con 0x7f54f815e7b0 2024-09-13T16:51:38.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f806c290 con 0x7f54f81407b0 2024-09-13T16:51:38.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.668+0000 7f54fd800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f8144010 con 0x7f54f815eb90 2024-09-13T16:51:38.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f8143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f8143490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40578/0 (socket says 172.21.15.3:40578) 2024-09-13T16:51:38.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 -- 172.21.15.3:0/3029702287 learned_addr learned my addr 172.21.15.3:0/3029702287 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:38.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7e00700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f81407b0 0x7f54f813d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54fce00700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f815eb90 0x7f54f81439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 -- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f815eb90 msgr2=0x7f54f81439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f815eb90 0x7f54f81439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 -- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f81407b0 msgr2=0x7f54f813d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f81407b0 0x7f54f813d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 -- 172.21.15.3:0/3029702287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40094b0 con 0x7f54f815e7b0 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54fce00700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f815eb90 0x7f54f81439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54f7400700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f8143490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f54e400ebe0 tx=0x7f54e400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54efe00700 1 -- 172.21.15.3:0/3029702287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400c850 con 0x7f54f815e7b0 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54efe00700 1 -- 172.21.15.3:0/3029702287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e400c9b0 con 0x7f54f815e7b0 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e8009570 con 0x7f54f815e7b0 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54efe00700 1 -- 172.21.15.3:0/3029702287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400cd40 con 0x7f54f815e7b0 2024-09-13T16:51:38.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f806c4a0 con 0x7f54f815e7b0 2024-09-13T16:51:38.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.672+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f813e810 con 0x7f54f815e7b0 2024-09-13T16:51:38.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.676+0000 7f54efe00700 1 -- 172.21.15.3:0/3029702287 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54e4013470 con 0x7f54f815e7b0 2024-09-13T16:51:38.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.676+0000 7f54efe00700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f54d8066730 0x7f54d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:38.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.676+0000 7f54efe00700 1 -- 172.21.15.3:0/3029702287 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f54e4084610 con 0x7f54f815e7b0 2024-09-13T16:51:38.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.680+0000 7f54f7e00700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f54d8066730 0x7f54d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:38.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.680+0000 7f54f7e00700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f54d8066730 0x7f54d8068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c00 tx=0x7f54e8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:38.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.684+0000 7f54efe00700 1 -- 172.21.15.3:0/3029702287 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f54e40515b0 con 0x7f54f815e7b0 2024-09-13T16:51:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.864+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f54f806c070 con 0x7f54f815e7b0 2024-09-13T16:51:38.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.864+0000 7f54efe00700 1 -- 172.21.15.3:0/3029702287 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f54e400cea0 con 0x7f54f815e7b0 2024-09-13T16:51:38.870 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f54d8066730 msgr2=0x7f54d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f54d8066730 0x7f54d8068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c00 tx=0x7f54e8008030 comp rx=0 tx=0).stop 2024-09-13T16:51:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 msgr2=0x7f54f8143490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f8143490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f54e400ebe0 tx=0x7f54e400eef0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 shutdown_connections 2024-09-13T16:51:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f54d8066730 0x7f54d8068bf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f815eb90 0x7f54f81439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54f815e7b0 0x7f54f8143490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 --2- 172.21.15.3:0/3029702287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f81407b0 0x7f54f813d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 >> 172.21.15.3:0/3029702287 conn(0x7f54f80712c0 msgr2=0x7f54f813f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 shutdown_connections 2024-09-13T16:51:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:38.868+0000 7f54fd800700 1 -- 172.21.15.3:0/3029702287 wait complete. 2024-09-13T16:51:38.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:51:39.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.064+0000 7fe6f5c00700 1 Processor -- start 2024-09-13T16:51:39.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.064+0000 7fe6f5c00700 1 -- start start 2024-09-13T16:51:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6f006ccf0 0x7fe6f01472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6f006fea0 0x7fe6f0070310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f0070850 con 0x7fe6f0146ee0 2024-09-13T16:51:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f0070a80 con 0x7fe6f006fea0 2024-09-13T16:51:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f01626e0 con 0x7fe6f006ccf0 2024-09-13T16:51:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6f006fea0 0x7fe6f0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6efe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe6f0146ee0 0x7fe6f006c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37446/0 (socket says 172.21.15.3:37446) 2024-09-13T16:51:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6efe00700 1 -- 172.21.15.3:0/3850447776 learned_addr learned my addr 172.21.15.3:0/3850447776 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6e7e00700 1 --2- 172.21.15.3:0/3850447776 >> v2:172.21.15.157:3300/0 conn(0x7fe6f006ccf0 0x7fe6f01472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5200700 1 -- 172.21.15.3:0/3850447776 >> v2:172.21.15.157:3300/0 conn(0x7fe6f006ccf0 msgr2=0x7fe6f01472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5200700 1 --2- 172.21.15.3:0/3850447776 >> v2:172.21.15.157:3300/0 conn(0x7fe6f006ccf0 0x7fe6f01472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5200700 1 -- 172.21.15.3:0/3850447776 >> v1:172.21.15.3:6789/0 conn(0x7fe6f0146ee0 legacy=0x7fe6f006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5200700 1 -- 172.21.15.3:0/3850447776 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0165800 con 0x7fe6f006fea0 2024-09-13T16:51:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5200700 1 --2- 172.21.15.3:0/3850447776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f006fea0 0x7fe6f0070310 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6f006b830 tx=0x7fe6e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d75064026089914 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6ef400700 1 -- 172.21.15.3:0/3850447776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e800e070 con 0x7fe6f006fea0 2024-09-13T16:51:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6ef400700 1 -- 172.21.15.3:0/3850447776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e800bbd0 con 0x7fe6f006fea0 2024-09-13T16:51:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6ef400700 1 -- 172.21.15.3:0/3850447776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8012430 con 0x7fe6f006fea0 2024-09-13T16:51:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5c00700 1 -- 172.21.15.3:0/3850447776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f006fea0 msgr2=0x7fe6f0070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.068+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/3850447776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f006fea0 0x7fe6f0070310 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6f006b830 tx=0x7fe6e800b330 comp rx=0 tx=0).stop 2024-09-13T16:51:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.072+0000 7fe6f5c00700 1 -- 172.21.15.3:0/3850447776 shutdown_connections 2024-09-13T16:51:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.072+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/3850447776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f006fea0 0x7fe6f0070310 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.072+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/3850447776 >> v2:172.21.15.157:3300/0 conn(0x7fe6f006ccf0 0x7fe6f01472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.072+0000 7fe6f5c00700 1 -- 172.21.15.3:0/3850447776 >> 172.21.15.3:0/3850447776 conn(0x7fe6f00b0cf0 msgr2=0x7fe6f00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.072+0000 7fe6f5c00700 1 -- 172.21.15.3:0/3850447776 shutdown_connections 2024-09-13T16:51:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.072+0000 7fe6f5c00700 1 -- 172.21.15.3:0/3850447776 wait complete. 2024-09-13T16:51:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.072+0000 7fe6f5c00700 1 Processor -- start 2024-09-13T16:51:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.072+0000 7fe6f5c00700 1 -- start start 2024-09-13T16:51:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f006ccf0 0x7fe6f0143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f006fea0 0x7fe6f0161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f0146ee0 0x7fe6f00b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f0143d60 con 0x7fe6f0146ee0 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f0161b40 con 0x7fe6f006fea0 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f0161e20 con 0x7fe6f006ccf0 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f0146ee0 0x7fe6f00b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f0146ee0 0x7fe6f00b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40584/0 (socket says 172.21.15.3:40584) 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 -- 172.21.15.3:0/670799297 learned_addr learned my addr 172.21.15.3:0/670799297 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6e7e00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f006fea0 0x7fe6f0161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6efe00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f006ccf0 0x7fe6f0143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 -- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f006ccf0 msgr2=0x7fe6f0143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f006ccf0 0x7fe6f0143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 -- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f006fea0 msgr2=0x7fe6f0161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f006fea0 0x7fe6f0161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 -- 172.21.15.3:0/670799297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8009a70 con 0x7fe6f0146ee0 2024-09-13T16:51:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6efe00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f006ccf0 0x7fe6f0143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5200700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f0146ee0 0x7fe6f00b2c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8014040 tx=0x7fe6e8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6ecc00700 1 -- 172.21.15.3:0/670799297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8012430 con 0x7fe6f0146ee0 2024-09-13T16:51:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6ecc00700 1 -- 172.21.15.3:0/670799297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e8020070 con 0x7fe6f0146ee0 2024-09-13T16:51:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f0162100 con 0x7fe6f0146ee0 2024-09-13T16:51:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6ecc00700 1 -- 172.21.15.3:0/670799297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8004280 con 0x7fe6f0146ee0 2024-09-13T16:51:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.076+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f0161680 con 0x7fe6f0146ee0 2024-09-13T16:51:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.080+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f0162340 con 0x7fe6f0146ee0 2024-09-13T16:51:39.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.080+0000 7fe6ecc00700 1 -- 172.21.15.3:0/670799297 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6e80043e0 con 0x7fe6f0146ee0 2024-09-13T16:51:39.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.084+0000 7fe6ecc00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6cc066730 0x7fe6cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.084+0000 7fe6ecc00700 1 -- 172.21.15.3:0/670799297 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe6e8022070 con 0x7fe6f0146ee0 2024-09-13T16:51:39.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.084+0000 7fe6efe00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6cc066730 0x7fe6cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.084+0000 7fe6efe00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6cc066730 0x7fe6cc068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0007fd0 tx=0x7fe6e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.088+0000 7fe6ecc00700 1 -- 172.21.15.3:0/670799297 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6f0162340 con 0x7fe6f0146ee0 2024-09-13T16:51:39.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.280+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe6f01449d0 con 0x7fe6f0146ee0 2024-09-13T16:51:39.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.280+0000 7fe6ecc00700 1 -- 172.21.15.3:0/670799297 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe6f01449d0 con 0x7fe6f0146ee0 2024-09-13T16:51:39.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6cc066730 msgr2=0x7fe6cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6cc066730 0x7fe6cc068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0007fd0 tx=0x7fe6e0009040 comp rx=0 tx=0).stop 2024-09-13T16:51:39.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f0146ee0 msgr2=0x7fe6f00b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f0146ee0 0x7fe6f00b2c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8014040 tx=0x7fe6e8004af0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 shutdown_connections 2024-09-13T16:51:39.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6cc066730 0x7fe6cc068bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f0146ee0 0x7fe6f00b2c20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f006fea0 0x7fe6f0161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 --2- 172.21.15.3:0/670799297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f006ccf0 0x7fe6f0143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 >> 172.21.15.3:0/670799297 conn(0x7fe6f00b0cf0 msgr2=0x7fe6f00cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:39.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 shutdown_connections 2024-09-13T16:51:39.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.284+0000 7fe6f5c00700 1 -- 172.21.15.3:0/670799297 wait complete. 2024-09-13T16:51:39.293 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-13T16:51:39.316 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-13T16:51:39.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:39.316 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-13T16:51:39.372 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:51:39.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:39.372 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T16:51:39.407 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:51:39.407 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:39.423 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:51:39.423 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:39.485 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:51:39.485 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:39.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.756+0000 7f470d400700 1 Processor -- start 2024-09-13T16:51:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.756+0000 7f470d400700 1 -- start start 2024-09-13T16:51:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f470d400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f470806fb50 0x7f47080cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f470d400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47080b4560 0x7f470815a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f470d400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47080cf070 con 0x7f470806ff30 2024-09-13T16:51:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f470d400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47080b4be0 con 0x7f470806fb50 2024-09-13T16:51:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f470d400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47080cf330 con 0x7f47080b4560 2024-09-13T16:51:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4707e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f470806fb50 0x7f47080cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4707400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f470806ff30 0x7f47080b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37450/0 (socket says 172.21.15.3:37450) 2024-09-13T16:51:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4707400700 1 -- 172.21.15.3:0/3648170831 learned_addr learned my addr 172.21.15.3:0/3648170831 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f470ca00700 1 --2- 172.21.15.3:0/3648170831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47080b4560 0x7f470815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4706a00700 1 -- 172.21.15.3:0/3648170831 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373070521 0 0) 0x7f47080cf070 con 0x7f470806ff30 2024-09-13T16:51:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4706a00700 1 -- 172.21.15.3:0/3648170831 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ec0035d0 con 0x7f470806ff30 2024-09-13T16:51:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4707e00700 1 -- 172.21.15.3:0/3648170831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47080b4560 msgr2=0x7f470815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4707e00700 1 --2- 172.21.15.3:0/3648170831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47080b4560 0x7f470815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4707e00700 1 -- 172.21.15.3:0/3648170831 >> v1:172.21.15.3:6789/0 conn(0x7f470806ff30 legacy=0x7f47080b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4707e00700 1 -- 172.21.15.3:0/3648170831 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47080cf610 con 0x7f470806fb50 2024-09-13T16:51:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4707e00700 1 --2- 172.21.15.3:0/3648170831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806fb50 0x7f47080cfe10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009850 tx=0x7f46f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536221aad199988f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4706a00700 1 -- 172.21.15.3:0/3648170831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8011830 con 0x7f470806fb50 2024-09-13T16:51:39.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4706a00700 1 -- 172.21.15.3:0/3648170831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8011990 con 0x7f470806fb50 2024-09-13T16:51:39.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.760+0000 7f4706a00700 1 -- 172.21.15.3:0/3648170831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8011d40 con 0x7f470806fb50 2024-09-13T16:51:39.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.764+0000 7f470d400700 1 -- 172.21.15.3:0/3648170831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806fb50 msgr2=0x7f47080cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.764+0000 7f470d400700 1 --2- 172.21.15.3:0/3648170831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806fb50 0x7f47080cfe10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009850 tx=0x7f46f8009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:39.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.764+0000 7f470d400700 1 -- 172.21.15.3:0/3648170831 shutdown_connections 2024-09-13T16:51:39.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.764+0000 7f470d400700 1 --2- 172.21.15.3:0/3648170831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47080b4560 0x7f470815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.764+0000 7f470d400700 1 --2- 172.21.15.3:0/3648170831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806fb50 0x7f47080cfe10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.764+0000 7f470d400700 1 -- 172.21.15.3:0/3648170831 >> 172.21.15.3:0/3648170831 conn(0x7f47080b2910 msgr2=0x7f47080b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:39.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.764+0000 7f470d400700 1 -- 172.21.15.3:0/3648170831 shutdown_connections 2024-09-13T16:51:39.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.764+0000 7f470d400700 1 -- 172.21.15.3:0/3648170831 wait complete. 2024-09-13T16:51:39.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470d400700 1 Processor -- start 2024-09-13T16:51:39.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470d400700 1 -- start start 2024-09-13T16:51:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470d400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f470806fb50 0x7f47081468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806ff30 0x7f4708144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47080b4560 0x7f47080c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470d400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708144890 con 0x7f470806ff30 2024-09-13T16:51:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470d400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47080cc420 con 0x7f47080b4560 2024-09-13T16:51:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470d400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4708146f00 con 0x7f470806fb50 2024-09-13T16:51:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806ff30 0x7f4708144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806ff30 0x7f4708144170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40610/0 (socket says 172.21.15.3:40610) 2024-09-13T16:51:39.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707400700 1 -- 172.21.15.3:0/194544752 learned_addr learned my addr 172.21.15.3:0/194544752 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:39.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707e00700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f470806fb50 0x7f47081468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f470ca00700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47080b4560 0x7f47080c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707400700 1 -- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f470806fb50 msgr2=0x7f47081468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f470806fb50 0x7f47081468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707400700 1 -- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47080b4560 msgr2=0x7f47080c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47080b4560 0x7f47080c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.768+0000 7f4707400700 1 -- 172.21.15.3:0/194544752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f80094b0 con 0x7f470806ff30 2024-09-13T16:51:39.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.772+0000 7f4707400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806ff30 0x7f4708144170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c700 tx=0x7f46f400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:39.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.772+0000 7f46ffe00700 1 -- 172.21.15.3:0/194544752 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4005d90 con 0x7f470806ff30 2024-09-13T16:51:39.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.772+0000 7f46ffe00700 1 -- 172.21.15.3:0/194544752 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f40070d0 con 0x7f470806ff30 2024-09-13T16:51:39.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.772+0000 7f46ffe00700 1 -- 172.21.15.3:0/194544752 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4007480 con 0x7f470806ff30 2024-09-13T16:51:39.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.772+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47081471e0 con 0x7f470806ff30 2024-09-13T16:51:39.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.772+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47080c7e90 con 0x7f470806ff30 2024-09-13T16:51:39.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.772+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47080cc280 con 0x7f470806ff30 2024-09-13T16:51:39.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.776+0000 7f46ffe00700 1 -- 172.21.15.3:0/194544752 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f46f40075e0 con 0x7f470806ff30 2024-09-13T16:51:39.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.776+0000 7f46ffe00700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f46e8066730 0x7f46e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:39.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.776+0000 7f46ffe00700 1 -- 172.21.15.3:0/194544752 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f46f40839a0 con 0x7f470806ff30 2024-09-13T16:51:39.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.776+0000 7f4707e00700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f46e8066730 0x7f46e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:39.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.780+0000 7f4707e00700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f46e8066730 0x7f46e8068bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f46f80090b0 tx=0x7f46f8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:39.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.784+0000 7f46ffe00700 1 -- 172.21.15.3:0/194544752 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f47080cc280 con 0x7f470806ff30 2024-09-13T16:51:39.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.972+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47080cc280 con 0x7f470806ff30 2024-09-13T16:51:39.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.976+0000 7f46ffe00700 1 -- 172.21.15.3:0/194544752 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f47080cc280 con 0x7f470806ff30 2024-09-13T16:51:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:39.981 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f46e8066730 msgr2=0x7f46e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f46e8066730 0x7f46e8068bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f46f80090b0 tx=0x7f46f8008030 comp rx=0 tx=0).stop 2024-09-13T16:51:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806ff30 msgr2=0x7f4708144170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806ff30 0x7f4708144170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c700 tx=0x7f46f400ca10 comp rx=0 tx=0).stop 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 shutdown_connections 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f46e8066730 0x7f46e8068bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47080b4560 0x7f47080c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f470806ff30 0x7f4708144170 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 --2- 172.21.15.3:0/194544752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f470806fb50 0x7f47081468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 >> 172.21.15.3:0/194544752 conn(0x7f47080b2910 msgr2=0x7f47080b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 shutdown_connections 2024-09-13T16:51:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:39.980+0000 7f470d400700 1 -- 172.21.15.3:0/194544752 wait complete. 2024-09-13T16:51:40.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:40.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:40.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe361800700 1 Processor -- start 2024-09-13T16:51:40.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe361800700 1 -- start start 2024-09-13T16:51:40.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe361800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c070080 0x7fe35c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe361800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0b4d50 con 0x7fe35c06f0e0 2024-09-13T16:51:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe361800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0cfde0 con 0x7fe35c15ab10 2024-09-13T16:51:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe361800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c06f550 con 0x7fe35c070080 2024-09-13T16:51:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe35a000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe35c06f0e0 0x7fe35c06f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37452/0 (socket says 172.21.15.3:37452) 2024-09-13T16:51:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe35a000700 1 -- 172.21.15.3:0/1873972726 learned_addr learned my addr 172.21.15.3:0/1873972726 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.184+0000 7fe35b400700 1 --2- 172.21.15.3:0/1873972726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c070080 0x7fe35c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe359600700 1 -- 172.21.15.3:0/1873972726 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48458964 0 0) 0x7fe35c0b4d50 con 0x7fe35c06f0e0 2024-09-13T16:51:40.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe359600700 1 -- 172.21.15.3:0/1873972726 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3400035d0 con 0x7fe35c06f0e0 2024-09-13T16:51:40.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe359600700 1 -- 172.21.15.3:0/1873972726 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133967294 0 0) 0x7fe35c0cfde0 con 0x7fe35c15ab10 2024-09-13T16:51:40.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe35b400700 1 -- 172.21.15.3:0/1873972726 >> v1:172.21.15.157:6789/0 conn(0x7fe35c15ab10 legacy=0x7fe35c0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe35b400700 1 -- 172.21.15.3:0/1873972726 >> v1:172.21.15.3:6789/0 conn(0x7fe35c06f0e0 legacy=0x7fe35c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe35b400700 1 -- 172.21.15.3:0/1873972726 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0b4f80 con 0x7fe35c070080 2024-09-13T16:51:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe35b400700 1 --2- 172.21.15.3:0/1873972726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c070080 0x7fe35c0b4810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe354009dc0 tx=0x7fe35400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd2d5244e045cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe359600700 1 -- 172.21.15.3:0/1873972726 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400e070 con 0x7fe35c070080 2024-09-13T16:51:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe359600700 1 -- 172.21.15.3:0/1873972726 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35400bb20 con 0x7fe35c070080 2024-09-13T16:51:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe359600700 1 -- 172.21.15.3:0/1873972726 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400bed0 con 0x7fe35c070080 2024-09-13T16:51:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe361800700 1 -- 172.21.15.3:0/1873972726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c070080 msgr2=0x7fe35c0b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.188+0000 7fe361800700 1 --2- 172.21.15.3:0/1873972726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c070080 0x7fe35c0b4810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe354009dc0 tx=0x7fe35400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.192+0000 7fe361800700 1 -- 172.21.15.3:0/1873972726 shutdown_connections 2024-09-13T16:51:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.192+0000 7fe361800700 1 --2- 172.21.15.3:0/1873972726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c070080 0x7fe35c0b4810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.192+0000 7fe361800700 1 -- 172.21.15.3:0/1873972726 >> 172.21.15.3:0/1873972726 conn(0x7fe35c0b2b00 msgr2=0x7fe35c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.192+0000 7fe361800700 1 -- 172.21.15.3:0/1873972726 shutdown_connections 2024-09-13T16:51:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.192+0000 7fe361800700 1 -- 172.21.15.3:0/1873972726 wait complete. 2024-09-13T16:51:40.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.192+0000 7fe361800700 1 Processor -- start 2024-09-13T16:51:40.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.192+0000 7fe361800700 1 -- start start 2024-09-13T16:51:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe361800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c06f0e0 0x7fe35c14df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe361800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe35c070080 0x7fe35c154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe361800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c15ab10 0x7fe35c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe361800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c14e4a0 con 0x7fe35c06f0e0 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe361800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15d390 con 0x7fe35c070080 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe361800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15d670 con 0x7fe35c15ab10 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c06f0e0 0x7fe35c14df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c06f0e0 0x7fe35c14df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40624/0 (socket says 172.21.15.3:40624) 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 -- 172.21.15.3:0/1909326788 learned_addr learned my addr 172.21.15.3:0/1909326788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35b400700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c15ab10 0x7fe35c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35a000700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe35c070080 0x7fe35c154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 -- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c15ab10 msgr2=0x7fe35c15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c15ab10 0x7fe35c15af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 -- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe35c070080 msgr2=0x7fe35c154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe35c070080 0x7fe35c154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 -- 172.21.15.3:0/1909326788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe354009a70 con 0x7fe35c06f0e0 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35b400700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c15ab10 0x7fe35c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe35aa00700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c06f0e0 0x7fe35c14df60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00c520 tx=0x7fe34c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe352a00700 1 -- 172.21.15.3:0/1909326788 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0054e0 con 0x7fe35c06f0e0 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe352a00700 1 -- 172.21.15.3:0/1909326788 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0036a0 con 0x7fe35c06f0e0 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe352a00700 1 -- 172.21.15.3:0/1909326788 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003a50 con 0x7fe35c06f0e0 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c15d950 con 0x7fe35c06f0e0 2024-09-13T16:51:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.196+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c15df10 con 0x7fe35c06f0e0 2024-09-13T16:51:40.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.200+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c0b4810 con 0x7fe35c06f0e0 2024-09-13T16:51:40.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.200+0000 7fe352a00700 1 -- 172.21.15.3:0/1909326788 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe34c003bb0 con 0x7fe35c06f0e0 2024-09-13T16:51:40.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.200+0000 7fe352a00700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe33c066730 0x7fe33c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.200+0000 7fe352a00700 1 -- 172.21.15.3:0/1909326788 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe34c083970 con 0x7fe35c06f0e0 2024-09-13T16:51:40.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.204+0000 7fe35a000700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe33c066730 0x7fe33c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.204+0000 7fe35a000700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe33c066730 0x7fe33c068bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe35c06d1e0 tx=0x7fe348009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:40.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.208+0000 7fe352a00700 1 -- 172.21.15.3:0/1909326788 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe34c050880 con 0x7fe35c06f0e0 2024-09-13T16:51:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.392+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe35c1557b0 con 0x7fe35c06f0e0 2024-09-13T16:51:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.392+0000 7fe352a00700 1 -- 172.21.15.3:0/1909326788 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe34c0170a0 con 0x7fe35c06f0e0 2024-09-13T16:51:40.398 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe33c066730 msgr2=0x7fe33c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe33c066730 0x7fe33c068bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe35c06d1e0 tx=0x7fe348009040 comp rx=0 tx=0).stop 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c06f0e0 msgr2=0x7fe35c14df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c06f0e0 0x7fe35c14df60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00c520 tx=0x7fe34c00c830 comp rx=0 tx=0).stop 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 shutdown_connections 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe33c066730 0x7fe33c068bf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe35c15ab10 0x7fe35c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe35c070080 0x7fe35c154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 --2- 172.21.15.3:0/1909326788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c06f0e0 0x7fe35c14df60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 >> 172.21.15.3:0/1909326788 conn(0x7fe35c0b2b00 msgr2=0x7fe35c0b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 shutdown_connections 2024-09-13T16:51:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.396+0000 7fe361800700 1 -- 172.21.15.3:0/1909326788 wait complete. 2024-09-13T16:51:40.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:40.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5131200700 1 Processor -- start 2024-09-13T16:51:40.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5131200700 1 -- start start 2024-09-13T16:51:40.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5131200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c15ab10 0x7f512c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5131200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f512c06f970 0x7f512c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5131200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0cfde0 con 0x7f512c0b4810 2024-09-13T16:51:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5131200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c070340 con 0x7f512c06f970 2024-09-13T16:51:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5131200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0b4f80 con 0x7f512c15ab10 2024-09-13T16:51:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5129400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f512c0b4810 0x7f512c0d0270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37458/0 (socket says 172.21.15.3:37458) 2024-09-13T16:51:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.596+0000 7f5129400700 1 -- 172.21.15.3:0/1542250561 learned_addr learned my addr 172.21.15.3:0/1542250561 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:40.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5129e00700 1 --2- 172.21.15.3:0/1542250561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c15ab10 0x7f512c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f512a800700 1 --2- 172.21.15.3:0/1542250561 >> v2:172.21.15.157:3300/0 conn(0x7f512c06f970 0x7f512c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5128a00700 1 -- 172.21.15.3:0/1542250561 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448283956 0 0) 0x7f512c0cfde0 con 0x7f512c0b4810 2024-09-13T16:51:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5128a00700 1 -- 172.21.15.3:0/1542250561 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51100035d0 con 0x7f512c0b4810 2024-09-13T16:51:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5129e00700 1 -- 172.21.15.3:0/1542250561 >> v2:172.21.15.157:3300/0 conn(0x7f512c06f970 msgr2=0x7f512c06fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5129e00700 1 --2- 172.21.15.3:0/1542250561 >> v2:172.21.15.157:3300/0 conn(0x7f512c06f970 0x7f512c06fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5129e00700 1 -- 172.21.15.3:0/1542250561 >> v1:172.21.15.3:6789/0 conn(0x7f512c0b4810 legacy=0x7f512c0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5129e00700 1 -- 172.21.15.3:0/1542250561 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0cf790 con 0x7f512c15ab10 2024-09-13T16:51:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5129e00700 1 --2- 172.21.15.3:0/1542250561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c15ab10 0x7f512c06f0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f511c009d90 tx=0x7f511c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d92702e7bbea61d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5128a00700 1 -- 172.21.15.3:0/1542250561 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c00e070 con 0x7f512c15ab10 2024-09-13T16:51:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5128a00700 1 -- 172.21.15.3:0/1542250561 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c00bb20 con 0x7f512c15ab10 2024-09-13T16:51:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5128a00700 1 -- 172.21.15.3:0/1542250561 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c012430 con 0x7f512c15ab10 2024-09-13T16:51:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5131200700 1 -- 172.21.15.3:0/1542250561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c15ab10 msgr2=0x7f512c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5131200700 1 --2- 172.21.15.3:0/1542250561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c15ab10 0x7f512c06f0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f511c009d90 tx=0x7f511c00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5131200700 1 -- 172.21.15.3:0/1542250561 shutdown_connections 2024-09-13T16:51:40.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5131200700 1 --2- 172.21.15.3:0/1542250561 >> v2:172.21.15.157:3300/0 conn(0x7f512c06f970 0x7f512c06fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5131200700 1 --2- 172.21.15.3:0/1542250561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c15ab10 0x7f512c06f0e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5131200700 1 -- 172.21.15.3:0/1542250561 >> 172.21.15.3:0/1542250561 conn(0x7f512c0b2b00 msgr2=0x7f512c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:40.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5131200700 1 -- 172.21.15.3:0/1542250561 shutdown_connections 2024-09-13T16:51:40.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.600+0000 7f5131200700 1 -- 172.21.15.3:0/1542250561 wait complete. 2024-09-13T16:51:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.604+0000 7f5131200700 1 Processor -- start 2024-09-13T16:51:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.604+0000 7f5131200700 1 -- start start 2024-09-13T16:51:40.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.604+0000 7f5131200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c06f970 0x7f512c150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.604+0000 7f5131200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f512c0b4810 0x7f512c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.604+0000 7f5131200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f512c15ab10 0x7f512c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.604+0000 7f5131200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c1f6ca0 con 0x7f512c15ab10 2024-09-13T16:51:40.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.604+0000 7f5131200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c1f6020 con 0x7f512c0b4810 2024-09-13T16:51:40.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.604+0000 7f5131200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c1f6300 con 0x7f512c06f970 2024-09-13T16:51:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f512c15ab10 0x7f512c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f512c15ab10 0x7f512c153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40630/0 (socket says 172.21.15.3:40630) 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 -- 172.21.15.3:0/3780942378 learned_addr learned my addr 172.21.15.3:0/3780942378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5129400700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f512c0b4810 0x7f512c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5129e00700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c06f970 0x7f512c150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 -- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c06f970 msgr2=0x7f512c150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c06f970 0x7f512c150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 -- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f512c0b4810 msgr2=0x7f512c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f512c0b4810 0x7f512c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 -- 172.21.15.3:0/3780942378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c009a70 con 0x7f512c15ab10 2024-09-13T16:51:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5129e00700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c06f970 0x7f512c150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f512a800700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f512c15ab10 0x7f512c153860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f512400eb60 tx=0x7f512400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5122000700 1 -- 172.21.15.3:0/3780942378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124009740 con 0x7f512c15ab10 2024-09-13T16:51:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5122000700 1 -- 172.21.15.3:0/3780942378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400c850 con 0x7f512c15ab10 2024-09-13T16:51:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5122000700 1 -- 172.21.15.3:0/3780942378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400cbe0 con 0x7f512c15ab10 2024-09-13T16:51:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c1f65e0 con 0x7f512c15ab10 2024-09-13T16:51:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c0b23e0 con 0x7f512c15ab10 2024-09-13T16:51:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.608+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c1f6de0 con 0x7f512c15ab10 2024-09-13T16:51:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.612+0000 7f5122000700 1 -- 172.21.15.3:0/3780942378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f512400cd40 con 0x7f512c15ab10 2024-09-13T16:51:40.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.612+0000 7f5122000700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f510c066730 0x7f510c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:40.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.612+0000 7f5122000700 1 -- 172.21.15.3:0/3780942378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5124084510 con 0x7f512c15ab10 2024-09-13T16:51:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.612+0000 7f5129e00700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f510c066730 0x7f510c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:40.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.616+0000 7f5129e00700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f510c066730 0x7f510c068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f511c014040 tx=0x7f511c011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:40.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.620+0000 7f5122000700 1 -- 172.21.15.3:0/3780942378 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f512c1f6de0 con 0x7f512c15ab10 2024-09-13T16:51:40.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.800+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f512c1f6de0 con 0x7f512c15ab10 2024-09-13T16:51:40.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5122000700 1 -- 172.21.15.3:0/3780942378 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f512c1f6de0 con 0x7f512c15ab10 2024-09-13T16:51:40.809 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:40.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f510c066730 msgr2=0x7f510c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f510c066730 0x7f510c068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f511c014040 tx=0x7f511c011000 comp rx=0 tx=0).stop 2024-09-13T16:51:40.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f512c15ab10 msgr2=0x7f512c153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:40.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f512c15ab10 0x7f512c153860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f512400eb60 tx=0x7f512400ef20 comp rx=0 tx=0).stop 2024-09-13T16:51:40.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 shutdown_connections 2024-09-13T16:51:40.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f510c066730 0x7f510c068bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f512c15ab10 0x7f512c153860 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f512c0b4810 0x7f512c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.804+0000 7f5131200700 1 --2- 172.21.15.3:0/3780942378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f512c06f970 0x7f512c150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:40.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.808+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 >> 172.21.15.3:0/3780942378 conn(0x7f512c0b2b00 msgr2=0x7f512c158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:40.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.808+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 shutdown_connections 2024-09-13T16:51:40.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:40.808+0000 7f5131200700 1 -- 172.21.15.3:0/3780942378 wait complete. 2024-09-13T16:51:40.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b7d400700 1 Processor -- start 2024-09-13T16:51:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b7d400700 1 -- start start 2024-09-13T16:51:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b7d400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b780b4ec0 0x7f0b78071960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b7d400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 0x7f0b780ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b7d400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780722e0 con 0x7f0b78071ea0 2024-09-13T16:51:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b7d400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7815abe0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b7d400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b52a0 con 0x7f0b7815a790 2024-09-13T16:51:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b76800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b780b4ec0 0x7f0b78071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b76800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b780b4ec0 0x7f0b78071960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:57794/0 (socket says 172.21.15.3:57794) 2024-09-13T16:51:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b76800700 1 -- 172.21.15.3:0/1067735511 learned_addr learned my addr 172.21.15.3:0/1067735511 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b77200700 1 --2- 172.21.15.3:0/1067735511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 0x7f0b780ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b76800700 1 -- 172.21.15.3:0/1067735511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 msgr2=0x7f0b780ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b76800700 1 --2- 172.21.15.3:0/1067735511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 0x7f0b780ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b76800700 1 -- 172.21.15.3:0/1067735511 >> v1:172.21.15.157:6789/0 conn(0x7f0b78071ea0 legacy=0x7f0b7815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.004+0000 7f0b76800700 1 -- 172.21.15.3:0/1067735511 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7815ae40 con 0x7f0b780b4ec0 2024-09-13T16:51:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b76800700 1 --2- 172.21.15.3:0/1067735511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b780b4ec0 0x7f0b78071960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800bbb0 tx=0x7f0b6800bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34623724669c5715 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b75400700 1 -- 172.21.15.3:0/1067735511 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800dd90 con 0x7f0b780b4ec0 2024-09-13T16:51:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b75400700 1 -- 172.21.15.3:0/1067735511 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68010470 con 0x7f0b780b4ec0 2024-09-13T16:51:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b75400700 1 -- 172.21.15.3:0/1067735511 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68010820 con 0x7f0b780b4ec0 2024-09-13T16:51:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b7d400700 1 -- 172.21.15.3:0/1067735511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b780b4ec0 msgr2=0x7f0b78071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b7d400700 1 --2- 172.21.15.3:0/1067735511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b780b4ec0 0x7f0b78071960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800bbb0 tx=0x7f0b6800bec0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b7d400700 1 -- 172.21.15.3:0/1067735511 shutdown_connections 2024-09-13T16:51:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b7d400700 1 --2- 172.21.15.3:0/1067735511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 0x7f0b780ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b7d400700 1 --2- 172.21.15.3:0/1067735511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b780b4ec0 0x7f0b78071960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b7d400700 1 -- 172.21.15.3:0/1067735511 >> 172.21.15.3:0/1067735511 conn(0x7f0b7806dff0 msgr2=0x7f0b78071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b7d400700 1 -- 172.21.15.3:0/1067735511 shutdown_connections 2024-09-13T16:51:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.008+0000 7f0b7d400700 1 -- 172.21.15.3:0/1067735511 wait complete. 2024-09-13T16:51:41.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.012+0000 7f0b7d400700 1 Processor -- start 2024-09-13T16:51:41.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.012+0000 7f0b7d400700 1 -- start start 2024-09-13T16:51:41.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.012+0000 7f0b7d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b78071ea0 0x7f0b78154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.012+0000 7f0b7d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b780b4ec0 0x7f0b7814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.012+0000 7f0b7d400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 0x7f0b780cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.012+0000 7f0b7d400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780caec0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.012+0000 7f0b7d400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780cb1a0 con 0x7f0b78071ea0 2024-09-13T16:51:41.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.012+0000 7f0b7d400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b781503c0 con 0x7f0b7815a790 2024-09-13T16:51:41.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b780b4ec0 0x7f0b7814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b780b4ec0 0x7f0b7814f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40638/0 (socket says 172.21.15.3:40638) 2024-09-13T16:51:41.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 -- 172.21.15.3:0/3262019487 learned_addr learned my addr 172.21.15.3:0/3262019487 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:41.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b77200700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 0x7f0b780cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b76800700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b78071ea0 0x7f0b78154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 -- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 msgr2=0x7f0b780cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 0x7f0b780cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 -- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b78071ea0 msgr2=0x7f0b78154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b78071ea0 0x7f0b78154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 -- 172.21.15.3:0/3262019487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6800b810 con 0x7f0b780b4ec0 2024-09-13T16:51:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b76800700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b78071ea0 0x7f0b78154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b75e00700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b780b4ec0 0x7f0b7814f5a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400c500 tx=0x7f0b6400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b6ea00700 1 -- 172.21.15.3:0/3262019487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640052d0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b6ea00700 1 -- 172.21.15.3:0/3262019487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b64005430 con 0x7f0b780b4ec0 2024-09-13T16:51:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b6ea00700 1 -- 172.21.15.3:0/3262019487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640055a0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b781506a0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.016+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7814fae0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.020+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b78150bc0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.020+0000 7f0b6ea00700 1 -- 172.21.15.3:0/3262019487 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b640038e0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.020+0000 7f0b6ea00700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0b58066760 0x7f0b58068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.024+0000 7f0b6ea00700 1 -- 172.21.15.3:0/3262019487 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0b64083920 con 0x7f0b780b4ec0 2024-09-13T16:51:41.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.024+0000 7f0b76800700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0b58066760 0x7f0b58068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.024+0000 7f0b76800700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0b58066760 0x7f0b58068c20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800bf00 tx=0x7f0b6801b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.028+0000 7f0b6ea00700 1 -- 172.21.15.3:0/3262019487 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0b640507a0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.212+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b780cb310 con 0x7f0b780b4ec0 2024-09-13T16:51:41.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.212+0000 7f0b6ea00700 1 -- 172.21.15.3:0/3262019487 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b640170a0 con 0x7f0b780b4ec0 2024-09-13T16:51:41.219 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:41.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0b58066760 msgr2=0x7f0b58068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0b58066760 0x7f0b58068c20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800bf00 tx=0x7f0b6801b040 comp rx=0 tx=0).stop 2024-09-13T16:51:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b780b4ec0 msgr2=0x7f0b7814f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b780b4ec0 0x7f0b7814f5a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400c500 tx=0x7f0b6400c810 comp rx=0 tx=0).stop 2024-09-13T16:51:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 shutdown_connections 2024-09-13T16:51:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0b58066760 0x7f0b58068c20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b7815a790 0x7f0b780cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b780b4ec0 0x7f0b7814f5a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 --2- 172.21.15.3:0/3262019487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b78071ea0 0x7f0b78154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 >> 172.21.15.3:0/3262019487 conn(0x7f0b7806dff0 msgr2=0x7f0b7806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 shutdown_connections 2024-09-13T16:51:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.216+0000 7f0b7d400700 1 -- 172.21.15.3:0/3262019487 wait complete. 2024-09-13T16:51:41.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.412+0000 7f05ffe00700 1 Processor -- start 2024-09-13T16:51:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.412+0000 7f05ffe00700 1 -- start start 2024-09-13T16:51:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05ffe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f060006d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05ffe00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06000b41e0 con 0x7f060006dcb0 2024-09-13T16:51:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05ffe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060006a390 con 0x7f06000cded0 2024-09-13T16:51:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05ffe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060006e0f0 con 0x7f06001442c0 2024-09-13T16:51:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05f6000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f060006dcb0 0x7f06000cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37460/0 (socket says 172.21.15.3:37460) 2024-09-13T16:51:41.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05f6000700 1 -- 172.21.15.3:0/3595215831 learned_addr learned my addr 172.21.15.3:0/3595215831 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:41.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05fea00700 1 --2- 172.21.15.3:0/3595215831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f060006d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05fea00700 1 -- 172.21.15.3:0/3595215831 >> v1:172.21.15.157:6789/0 conn(0x7f06000cded0 legacy=0x7f060015b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05fea00700 1 -- 172.21.15.3:0/3595215831 >> v1:172.21.15.3:6789/0 conn(0x7f060006dcb0 legacy=0x7f06000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05fea00700 1 -- 172.21.15.3:0/3595215831 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060006b1f0 con 0x7f06001442c0 2024-09-13T16:51:41.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.416+0000 7f05fea00700 1 --2- 172.21.15.3:0/3595215831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f060006d770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009880 tx=0x7f05f0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755453b1113584c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05fe000700 1 -- 172.21.15.3:0/3595215831 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0011830 con 0x7f06001442c0 2024-09-13T16:51:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05fe000700 1 -- 172.21.15.3:0/3595215831 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f0011990 con 0x7f06001442c0 2024-09-13T16:51:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05fe000700 1 -- 172.21.15.3:0/3595215831 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0011d20 con 0x7f06001442c0 2024-09-13T16:51:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05ffe00700 1 -- 172.21.15.3:0/3595215831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 msgr2=0x7f060006d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05ffe00700 1 --2- 172.21.15.3:0/3595215831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f060006d770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009880 tx=0x7f05f0009b90 comp rx=0 tx=0).stop 2024-09-13T16:51:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05ffe00700 1 -- 172.21.15.3:0/3595215831 shutdown_connections 2024-09-13T16:51:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05ffe00700 1 --2- 172.21.15.3:0/3595215831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f060006d770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05ffe00700 1 -- 172.21.15.3:0/3595215831 >> 172.21.15.3:0/3595215831 conn(0x7f06000af620 msgr2=0x7f06000afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05ffe00700 1 -- 172.21.15.3:0/3595215831 shutdown_connections 2024-09-13T16:51:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.420+0000 7f05ffe00700 1 -- 172.21.15.3:0/3595215831 wait complete. 2024-09-13T16:51:41.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ffe00700 1 Processor -- start 2024-09-13T16:51:41.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ffe00700 1 -- start start 2024-09-13T16:51:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ffe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f060006dcb0 0x7f060015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ffe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000cded0 0x7f06000acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ffe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f06000aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ffe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000ad2b0 con 0x7f060006dcb0 2024-09-13T16:51:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ffe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060013eac0 con 0x7f06000cded0 2024-09-13T16:51:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ffe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000ab410 con 0x7f06001442c0 2024-09-13T16:51:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f060006dcb0 0x7f060015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f060006dcb0 0x7f060015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40642/0 (socket says 172.21.15.3:40642) 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 -- 172.21.15.3:0/164546999 learned_addr learned my addr 172.21.15.3:0/164546999 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05f6000700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000cded0 0x7f06000acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ff400700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f06000aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 -- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 msgr2=0x7f06000aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f06000aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 -- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000cded0 msgr2=0x7f06000acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000cded0 0x7f06000acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 -- 172.21.15.3:0/164546999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f00094b0 con 0x7f060006dcb0 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05f6000700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000cded0 0x7f06000acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05ff400700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f06000aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.424+0000 7f05fea00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f060006dcb0 0x7f060015a5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f05f0014040 tx=0x7f05f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.428+0000 7f05f7e00700 1 -- 172.21.15.3:0/164546999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00038c0 con 0x7f060006dcb0 2024-09-13T16:51:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.428+0000 7f05f7e00700 1 -- 172.21.15.3:0/164546999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f001e070 con 0x7f060006dcb0 2024-09-13T16:51:41.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.428+0000 7f05f7e00700 1 -- 172.21.15.3:0/164546999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0003c80 con 0x7f060006dcb0 2024-09-13T16:51:41.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.428+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06000abcd0 con 0x7f060006dcb0 2024-09-13T16:51:41.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.428+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06000ac290 con 0x7f060006dcb0 2024-09-13T16:51:41.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.428+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0600144780 con 0x7f060006dcb0 2024-09-13T16:51:41.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.428+0000 7f05f7e00700 1 -- 172.21.15.3:0/164546999 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f05f001a420 con 0x7f060006dcb0 2024-09-13T16:51:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.432+0000 7f05f7e00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05dc066760 0x7f05dc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.432+0000 7f05f7e00700 1 -- 172.21.15.3:0/164546999 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f05f0020070 con 0x7f060006dcb0 2024-09-13T16:51:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.432+0000 7f05f6000700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05dc066760 0x7f05dc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.432+0000 7f05f6000700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05dc066760 0x7f05dc068c20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f05e8007fd0 tx=0x7f05e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.436+0000 7f05f7e00700 1 -- 172.21.15.3:0/164546999 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f05f0052b90 con 0x7f060006dcb0 2024-09-13T16:51:41.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.616+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f060015e300 con 0x7f060006dcb0 2024-09-13T16:51:41.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.616+0000 7f05f7e00700 1 -- 172.21.15.3:0/164546999 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f05f002a0a0 con 0x7f060006dcb0 2024-09-13T16:51:41.623 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:41.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05dc066760 msgr2=0x7f05dc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05dc066760 0x7f05dc068c20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f05e8007fd0 tx=0x7f05e8009040 comp rx=0 tx=0).stop 2024-09-13T16:51:41.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f060006dcb0 msgr2=0x7f060015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f060006dcb0 0x7f060015a5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f05f0014040 tx=0x7f05f0004770 comp rx=0 tx=0).stop 2024-09-13T16:51:41.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 shutdown_connections 2024-09-13T16:51:41.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f05dc066760 0x7f05dc068c20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06001442c0 0x7f06000aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000cded0 0x7f06000acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 --2- 172.21.15.3:0/164546999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f060006dcb0 0x7f060015a5d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 >> 172.21.15.3:0/164546999 conn(0x7f06000af620 msgr2=0x7f060015d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:41.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 shutdown_connections 2024-09-13T16:51:41.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.620+0000 7f05ffe00700 1 -- 172.21.15.3:0/164546999 wait complete. 2024-09-13T16:51:41.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:51:41.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb547e00700 1 Processor -- start 2024-09-13T16:51:41.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb547e00700 1 -- start start 2024-09-13T16:51:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb547e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b4b80 0x7fb5480ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb547e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5480ce8d0 0x7fb5481593e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb547e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54815aa00 con 0x7fb5480b47a0 2024-09-13T16:51:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb547e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480cdf70 con 0x7fb5480ce8d0 2024-09-13T16:51:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb547e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480cd980 con 0x7fb5480b4b80 2024-09-13T16:51:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb546a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb5480b47a0 0x7fb54806d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37470/0 (socket says 172.21.15.3:37470) 2024-09-13T16:51:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.828+0000 7fb546a00700 1 -- 172.21.15.3:0/356002960 learned_addr learned my addr 172.21.15.3:0/356002960 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb53e000700 1 --2- 172.21.15.3:0/356002960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b4b80 0x7fb5480ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb547400700 1 --2- 172.21.15.3:0/356002960 >> v2:172.21.15.157:3300/0 conn(0x7fb5480ce8d0 0x7fb5481593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb546000700 1 -- 172.21.15.3:0/356002960 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050870928 0 0) 0x7fb54815aa00 con 0x7fb5480b47a0 2024-09-13T16:51:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb546000700 1 -- 172.21.15.3:0/356002960 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb52c0035d0 con 0x7fb5480b47a0 2024-09-13T16:51:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb53e000700 1 -- 172.21.15.3:0/356002960 >> v2:172.21.15.157:3300/0 conn(0x7fb5480ce8d0 msgr2=0x7fb5481593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb53e000700 1 --2- 172.21.15.3:0/356002960 >> v2:172.21.15.157:3300/0 conn(0x7fb5480ce8d0 0x7fb5481593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb53e000700 1 -- 172.21.15.3:0/356002960 >> v1:172.21.15.3:6789/0 conn(0x7fb5480b47a0 legacy=0x7fb54806d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb53e000700 1 -- 172.21.15.3:0/356002960 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54815ace0 con 0x7fb5480b4b80 2024-09-13T16:51:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb547400700 1 --2- 172.21.15.3:0/356002960 >> v2:172.21.15.157:3300/0 conn(0x7fb5480ce8d0 0x7fb5481593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb53e000700 1 --2- 172.21.15.3:0/356002960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b4b80 0x7fb5480ce390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb530009850 tx=0x7fb530009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=697f12df85658790 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb546000700 1 -- 172.21.15.3:0/356002960 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530011830 con 0x7fb5480b4b80 2024-09-13T16:51:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb546000700 1 -- 172.21.15.3:0/356002960 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb530011990 con 0x7fb5480b4b80 2024-09-13T16:51:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb546000700 1 -- 172.21.15.3:0/356002960 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530011d40 con 0x7fb5480b4b80 2024-09-13T16:51:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb547e00700 1 -- 172.21.15.3:0/356002960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b4b80 msgr2=0x7fb5480ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.832+0000 7fb547e00700 1 --2- 172.21.15.3:0/356002960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b4b80 0x7fb5480ce390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb530009850 tx=0x7fb530009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.836+0000 7fb547e00700 1 -- 172.21.15.3:0/356002960 shutdown_connections 2024-09-13T16:51:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.836+0000 7fb547e00700 1 --2- 172.21.15.3:0/356002960 >> v2:172.21.15.157:3300/0 conn(0x7fb5480ce8d0 0x7fb5481593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.836+0000 7fb547e00700 1 --2- 172.21.15.3:0/356002960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b4b80 0x7fb5480ce390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.836+0000 7fb547e00700 1 -- 172.21.15.3:0/356002960 >> 172.21.15.3:0/356002960 conn(0x7fb548071da0 msgr2=0x7fb5480b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.836+0000 7fb547e00700 1 -- 172.21.15.3:0/356002960 shutdown_connections 2024-09-13T16:51:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.836+0000 7fb547e00700 1 -- 172.21.15.3:0/356002960 wait complete. 2024-09-13T16:51:41.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.836+0000 7fb547e00700 1 Processor -- start 2024-09-13T16:51:41.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547e00700 1 -- start start 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b47a0 0x7fb548157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5480b4b80 0x7fb5480b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5480ce8d0 0x7fb5480b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548157eb0 con 0x7fb5480b4b80 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5481f6620 con 0x7fb5480ce8d0 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480ad9a0 con 0x7fb5480b47a0 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5480b4b80 0x7fb5480b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5480b4b80 0x7fb5480b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40650/0 (socket says 172.21.15.3:40650) 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 -- 172.21.15.3:0/1106295355 learned_addr learned my addr 172.21.15.3:0/1106295355 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547400700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5480ce8d0 0x7fb5480b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb546a00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b47a0 0x7fb548157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 -- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b47a0 msgr2=0x7fb548157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b47a0 0x7fb548157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 -- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5480ce8d0 msgr2=0x7fb5480b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5480ce8d0 0x7fb5480b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 -- 172.21.15.3:0/1106295355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540009a70 con 0x7fb5480b4b80 2024-09-13T16:51:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb547400700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5480ce8d0 0x7fb5480b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.840+0000 7fb53e000700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5480b4b80 0x7fb5480b30f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb5300090b0 tx=0x7fb530004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.844+0000 7fb53f400700 1 -- 172.21.15.3:0/1106295355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300042c0 con 0x7fb5480b4b80 2024-09-13T16:51:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.844+0000 7fb53f400700 1 -- 172.21.15.3:0/1106295355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53001f070 con 0x7fb5480b4b80 2024-09-13T16:51:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.844+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5300094b0 con 0x7fb5480b4b80 2024-09-13T16:51:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.844+0000 7fb53f400700 1 -- 172.21.15.3:0/1106295355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300036c0 con 0x7fb5480b4b80 2024-09-13T16:51:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.844+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5480adf60 con 0x7fb5480b4b80 2024-09-13T16:51:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.844+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5480ce390 con 0x7fb5480b4b80 2024-09-13T16:51:41.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.848+0000 7fb53f400700 1 -- 172.21.15.3:0/1106295355 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb530021030 con 0x7fb5480b4b80 2024-09-13T16:51:41.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.848+0000 7fb53f400700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb524066730 0x7fb524068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:41.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.848+0000 7fb53f400700 1 -- 172.21.15.3:0/1106295355 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb530085820 con 0x7fb5480b4b80 2024-09-13T16:51:41.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.848+0000 7fb546a00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb524066730 0x7fb524068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:41.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.848+0000 7fb546a00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb524066730 0x7fb524068bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb538007fd0 tx=0x7fb538007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:41.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:41.856+0000 7fb53f400700 1 -- 172.21.15.3:0/1106295355 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb530052860 con 0x7fb5480b4b80 2024-09-13T16:51:42.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.056+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb5480b5b30 con 0x7fb5480b4b80 2024-09-13T16:51:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.056+0000 7fb53f400700 1 -- 172.21.15.3:0/1106295355 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb530006ce0 con 0x7fb5480b4b80 2024-09-13T16:51:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb524066730 msgr2=0x7fb524068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb524066730 0x7fb524068bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb538007fd0 tx=0x7fb538007de0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5480b4b80 msgr2=0x7fb5480b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5480b4b80 0x7fb5480b30f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb5300090b0 tx=0x7fb530004770 comp rx=0 tx=0).stop 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 shutdown_connections 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb524066730 0x7fb524068bf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5480ce8d0 0x7fb5480b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5480b4b80 0x7fb5480b30f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 --2- 172.21.15.3:0/1106295355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5480b47a0 0x7fb548157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 >> 172.21.15.3:0/1106295355 conn(0x7fb548071da0 msgr2=0x7fb54815b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 shutdown_connections 2024-09-13T16:51:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.060+0000 7fb547e00700 1 -- 172.21.15.3:0/1106295355 wait complete. 2024-09-13T16:51:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-13T16:51:42.086 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-13T16:51:42.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:42.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-13T16:51:42.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:51:42.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:42.125 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-13T16:51:42.187 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:51:42.187 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:42.205 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:51:42.205 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:42.265 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:51:42.265 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:51:42.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.508+0000 7f271a600700 1 Processor -- start 2024-09-13T16:51:42.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.508+0000 7f271a600700 1 -- start start 2024-09-13T16:51:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.508+0000 7f271a600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f27141443b0 0x7f27140b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.508+0000 7f271a600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2714144790 0x7f271406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.508+0000 7f271a600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27140ce8e0 con 0x7f271406cb20 2024-09-13T16:51:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.508+0000 7f271a600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271406cf90 con 0x7f27141443b0 2024-09-13T16:51:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.508+0000 7f271a600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27140ccd30 con 0x7f2714144790 2024-09-13T16:51:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2719200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f27141443b0 0x7f27140b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2713e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2714144790 0x7f271406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2719200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f27141443b0 0x7f27140b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40660/0 (socket says 172.21.15.3:40660) 2024-09-13T16:51:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2719200700 1 -- 172.21.15.3:0/2128528509 learned_addr learned my addr 172.21.15.3:0/2128528509 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2719200700 1 -- 172.21.15.3:0/2128528509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2714144790 msgr2=0x7f271406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2719200700 1 --2- 172.21.15.3:0/2128528509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2714144790 0x7f271406c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2719200700 1 -- 172.21.15.3:0/2128528509 >> v1:172.21.15.157:6789/0 conn(0x7f271406cb20 legacy=0x7f271406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2719200700 1 -- 172.21.15.3:0/2128528509 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27140cd010 con 0x7f27141443b0 2024-09-13T16:51:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2719200700 1 --2- 172.21.15.3:0/2128528509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140b39f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2708009910 tx=0x7f2708009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1e9cfb1731315e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2713400700 1 -- 172.21.15.3:0/2128528509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708011830 con 0x7f27141443b0 2024-09-13T16:51:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2713400700 1 -- 172.21.15.3:0/2128528509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2708011990 con 0x7f27141443b0 2024-09-13T16:51:42.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f2713400700 1 -- 172.21.15.3:0/2128528509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708011d40 con 0x7f27141443b0 2024-09-13T16:51:42.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f271a600700 1 -- 172.21.15.3:0/2128528509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 msgr2=0x7f27140b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f271a600700 1 --2- 172.21.15.3:0/2128528509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140b39f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2708009910 tx=0x7f2708009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f271a600700 1 -- 172.21.15.3:0/2128528509 shutdown_connections 2024-09-13T16:51:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f271a600700 1 --2- 172.21.15.3:0/2128528509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2714144790 0x7f271406c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f271a600700 1 --2- 172.21.15.3:0/2128528509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140b39f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f271a600700 1 -- 172.21.15.3:0/2128528509 >> 172.21.15.3:0/2128528509 conn(0x7f27140b0cf0 msgr2=0x7f27140b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f271a600700 1 -- 172.21.15.3:0/2128528509 shutdown_connections 2024-09-13T16:51:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.512+0000 7f271a600700 1 -- 172.21.15.3:0/2128528509 wait complete. 2024-09-13T16:51:42.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.516+0000 7f271a600700 1 Processor -- start 2024-09-13T16:51:42.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.516+0000 7f271a600700 1 -- start start 2024-09-13T16:51:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.516+0000 7f271a600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271406cb20 0x7f271413af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.516+0000 7f271a600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.516+0000 7f271a600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271413a6e0 0x7f27141f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.516+0000 7f271a600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271413b880 con 0x7f27141443b0 2024-09-13T16:51:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.516+0000 7f271a600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27140ae490 con 0x7f271406cb20 2024-09-13T16:51:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.516+0000 7f271a600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27140a9670 con 0x7f271413a6e0 2024-09-13T16:51:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40666/0 (socket says 172.21.15.3:40666) 2024-09-13T16:51:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 -- 172.21.15.3:0/3505839989 learned_addr learned my addr 172.21.15.3:0/3505839989 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2719200700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271406cb20 0x7f271413af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2719c00700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271413a6e0 0x7f27141f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 -- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271413a6e0 msgr2=0x7f27141f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271413a6e0 0x7f27141f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 -- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271406cb20 msgr2=0x7f271413af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271406cb20 0x7f271413af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 -- 172.21.15.3:0/3505839989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2708009570 con 0x7f27141443b0 2024-09-13T16:51:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2713e00700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140adf50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2700009fd0 tx=0x7f270000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2710c00700 1 -- 172.21.15.3:0/3505839989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000095a0 con 0x7f27141443b0 2024-09-13T16:51:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2710c00700 1 -- 172.21.15.3:0/3505839989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2700013680 con 0x7f27141443b0 2024-09-13T16:51:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27140a9950 con 0x7f27141443b0 2024-09-13T16:51:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f2710c00700 1 -- 172.21.15.3:0/3505839989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700013a30 con 0x7f27141443b0 2024-09-13T16:51:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27140a9f10 con 0x7f27141443b0 2024-09-13T16:51:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.520+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271406c5e0 con 0x7f27141443b0 2024-09-13T16:51:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.524+0000 7f2710c00700 1 -- 172.21.15.3:0/3505839989 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2700013b90 con 0x7f27141443b0 2024-09-13T16:51:42.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.524+0000 7f2710c00700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f26f4066730 0x7f26f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.524+0000 7f2710c00700 1 -- 172.21.15.3:0/3505839989 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f27000844d0 con 0x7f27141443b0 2024-09-13T16:51:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.524+0000 7f2719200700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f26f4066730 0x7f26f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.528+0000 7f2719200700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f26f4066730 0x7f26f4068bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2708000c00 tx=0x7f2708008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:42.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.532+0000 7f2710c00700 1 -- 172.21.15.3:0/3505839989 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2700051360 con 0x7f27141443b0 2024-09-13T16:51:42.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.728+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2714071d00 con 0x7f27141443b0 2024-09-13T16:51:42.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f2710c00700 1 -- 172.21.15.3:0/3505839989 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f2700054980 con 0x7f27141443b0 2024-09-13T16:51:42.736 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:42.737 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f26f4066730 msgr2=0x7f26f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f26f4066730 0x7f26f4068bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2708000c00 tx=0x7f2708008030 comp rx=0 tx=0).stop 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 msgr2=0x7f27140adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140adf50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2700009fd0 tx=0x7f270000eb20 comp rx=0 tx=0).stop 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 shutdown_connections 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f26f4066730 0x7f26f4068bf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271413a6e0 0x7f27141f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27141443b0 0x7f27140adf50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 --2- 172.21.15.3:0/3505839989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271406cb20 0x7f271413af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 >> 172.21.15.3:0/3505839989 conn(0x7f27140b0cf0 msgr2=0x7f27140b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 shutdown_connections 2024-09-13T16:51:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.732+0000 7f271a600700 1 -- 172.21.15.3:0/3505839989 wait complete. 2024-09-13T16:51:42.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:42.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:42.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.936+0000 7f034ea00700 1 Processor -- start 2024-09-13T16:51:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.936+0000 7f034ea00700 1 -- start start 2024-09-13T16:51:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f034ea00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03480b4510 0x7f034815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f034ea00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034815e170 con 0x7f0348153d80 2024-09-13T16:51:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f034ea00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034815a110 con 0x7f034815abf0 2024-09-13T16:51:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f034ea00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034815e450 con 0x7f03480b4510 2024-09-13T16:51:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f034d600700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0348153d80 0x7f03480b3e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37476/0 (socket says 172.21.15.3:37476) 2024-09-13T16:51:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f034d600700 1 -- 172.21.15.3:0/332116044 learned_addr learned my addr 172.21.15.3:0/332116044 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f034cc00700 1 --2- 172.21.15.3:0/332116044 >> v2:172.21.15.157:3300/0 conn(0x7f03480b4510 0x7f034815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269139232 0 0) 0x7f034815e170 con 0x7f0348153d80 2024-09-13T16:51:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03300035d0 con 0x7f0348153d80 2024-09-13T16:51:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4026608956 0 0) 0x7f03300035d0 con 0x7f0348153d80 2024-09-13T16:51:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 >> v2:172.21.15.157:3300/0 conn(0x7f03480b4510 msgr2=0x7f034815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 --2- 172.21.15.3:0/332116044 >> v2:172.21.15.157:3300/0 conn(0x7f03480b4510 0x7f034815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 >> v1:172.21.15.157:6789/0 conn(0x7f034815abf0 legacy=0x7f03480ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034815a3f0 con 0x7f0348153d80 2024-09-13T16:51:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f033c001670 con 0x7f0348153d80 2024-09-13T16:51:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.940+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 >> v1:172.21.15.3:6789/0 conn(0x7f0348153d80 legacy=0x7f03480b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0330003a20 0x7f0330003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330004870 0x7f0330004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0330005230 0x7f033000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034815e450 con 0x7f0330003a20 2024-09-13T16:51:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033000c0d0 con 0x7f0330005230 2024-09-13T16:51:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033000c290 con 0x7f0330004870 2024-09-13T16:51:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034d600700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0330003a20 0x7f0330003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034e000700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0330005230 0x7f033000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034cc00700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330004870 0x7f0330004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034d600700 1 -- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330004870 msgr2=0x7f0330004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034d600700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330004870 0x7f0330004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034d600700 1 -- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0330005230 msgr2=0x7f033000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034d600700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0330005230 0x7f033000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034d600700 1 -- 172.21.15.3:0/332116044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03300035d0 con 0x7f0330003a20 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034cc00700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330004870 0x7f0330004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034e000700 1 -- 172.21.15.3:0/332116044 reap_dead start 2024-09-13T16:51:42.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034d600700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0330003a20 0x7f0330003e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c320 tx=0x7f03300058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daef876bdb8cda48 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:42.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c007c30 con 0x7f0330003a20 2024-09-13T16:51:42.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c00ce80 con 0x7f0330003a20 2024-09-13T16:51:42.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f0347e00700 1 -- 172.21.15.3:0/332116044 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c005bc0 con 0x7f0330003a20 2024-09-13T16:51:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034ea00700 1 -- 172.21.15.3:0/332116044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0330003a20 msgr2=0x7f0330003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.944+0000 7f034ea00700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0330003a20 0x7f0330003e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c320 tx=0x7f03300058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 -- 172.21.15.3:0/332116044 shutdown_connections 2024-09-13T16:51:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 --2- 172.21.15.3:0/332116044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0330003a20 0x7f0330003e40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 -- 172.21.15.3:0/332116044 >> 172.21.15.3:0/332116044 conn(0x7f03480712c0 msgr2=0x7f03480716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 -- 172.21.15.3:0/332116044 shutdown_connections 2024-09-13T16:51:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 -- 172.21.15.3:0/332116044 wait complete. 2024-09-13T16:51:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 Processor -- start 2024-09-13T16:51:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 -- start start 2024-09-13T16:51:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330003a20 0x7f0348147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034814e700 0x7f034814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03480ce5c0 0x7f03480cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03480cef80 con 0x7f034814e700 2024-09-13T16:51:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.948+0000 7f034ea00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034814c0b0 con 0x7f03480ce5c0 2024-09-13T16:51:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034ea00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034814c390 con 0x7f0330003a20 2024-09-13T16:51:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034814e700 0x7f034814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034814e700 0x7f034814eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40696/0 (socket says 172.21.15.3:40696) 2024-09-13T16:51:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 -- 172.21.15.3:0/3150859530 learned_addr learned my addr 172.21.15.3:0/3150859530 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034d600700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330003a20 0x7f0348147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034e000700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03480ce5c0 0x7f03480cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 -- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330003a20 msgr2=0x7f0348147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330003a20 0x7f0348147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 -- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03480ce5c0 msgr2=0x7f03480cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03480ce5c0 0x7f03480cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 -- 172.21.15.3:0/3150859530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034814c670 con 0x7f034814e700 2024-09-13T16:51:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034e000700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03480ce5c0 0x7f03480cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034cc00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034814e700 0x7f034814eb80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0334011600 tx=0x7f03340119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f0345600700 1 -- 172.21.15.3:0/3150859530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334004220 con 0x7f034814e700 2024-09-13T16:51:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f0345600700 1 -- 172.21.15.3:0/3150859530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0334004380 con 0x7f034814e700 2024-09-13T16:51:42.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f0345600700 1 -- 172.21.15.3:0/3150859530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03340036c0 con 0x7f034814e700 2024-09-13T16:51:42.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034814c950 con 0x7f034814e700 2024-09-13T16:51:42.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.952+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034814d150 con 0x7f034814e700 2024-09-13T16:51:42.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.956+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03480b4190 con 0x7f034814e700 2024-09-13T16:51:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.956+0000 7f0345600700 1 -- 172.21.15.3:0/3150859530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0334003820 con 0x7f034814e700 2024-09-13T16:51:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.956+0000 7f0345600700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0328066760 0x7f0328068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.956+0000 7f0345600700 1 -- 172.21.15.3:0/3150859530 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f03340847f0 con 0x7f034814e700 2024-09-13T16:51:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.960+0000 7f034d600700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0328066760 0x7f0328068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:42.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.960+0000 7f034d600700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0328066760 0x7f0328068c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f033c000c00 tx=0x7f033c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:42.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:42.964+0000 7f0345600700 1 -- 172.21.15.3:0/3150859530 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0334051790 con 0x7f034814e700 2024-09-13T16:51:43.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.148+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f034806eb70 con 0x7f034814e700 2024-09-13T16:51:43.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.148+0000 7f0345600700 1 -- 172.21.15.3:0/3150859530 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0334022020 con 0x7f034814e700 2024-09-13T16:51:43.153 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:43.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0328066760 msgr2=0x7f0328068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0328066760 0x7f0328068c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f033c000c00 tx=0x7f033c007000 comp rx=0 tx=0).stop 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034814e700 msgr2=0x7f034814eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034814e700 0x7f034814eb80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0334011600 tx=0x7f03340119c0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 shutdown_connections 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0328066760 0x7f0328068c20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03480ce5c0 0x7f03480cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034814e700 0x7f034814eb80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 --2- 172.21.15.3:0/3150859530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0330003a20 0x7f0348147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 >> 172.21.15.3:0/3150859530 conn(0x7f03480712c0 msgr2=0x7f03480716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 shutdown_connections 2024-09-13T16:51:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.152+0000 7f034ea00700 1 -- 172.21.15.3:0/3150859530 wait complete. 2024-09-13T16:51:43.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.352+0000 7f6786e00700 1 Processor -- start 2024-09-13T16:51:43.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.352+0000 7f6786e00700 1 -- start start 2024-09-13T16:51:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6786e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678015ce20 0x7f67800cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6786e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67800ce240 0x7f67800b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6786e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6780161760 con 0x7f67800b3fe0 2024-09-13T16:51:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6786e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678015d200 con 0x7f67800ce240 2024-09-13T16:51:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6786e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67800b2a00 con 0x7f678015ce20 2024-09-13T16:51:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6785000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67800ce240 0x7f67800b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6786400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f67800b3fe0 0x7f6780160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37480/0 (socket says 172.21.15.3:37480) 2024-09-13T16:51:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6786400700 1 -- 172.21.15.3:0/2831566822 learned_addr learned my addr 172.21.15.3:0/2831566822 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6785a00700 1 --2- 172.21.15.3:0/2831566822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678015ce20 0x7f67800cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6785000700 1 -- 172.21.15.3:0/2831566822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678015ce20 msgr2=0x7f67800cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6785000700 1 --2- 172.21.15.3:0/2831566822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678015ce20 0x7f67800cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6785000700 1 -- 172.21.15.3:0/2831566822 >> v1:172.21.15.3:6789/0 conn(0x7f67800b3fe0 legacy=0x7f6780160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6785000700 1 -- 172.21.15.3:0/2831566822 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67800b2ce0 con 0x7f67800ce240 2024-09-13T16:51:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f6785000700 1 --2- 172.21.15.3:0/2831566822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67800ce240 0x7f67800b3aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f676c009850 tx=0x7f676c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2b8c09c1436fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f677fe00700 1 -- 172.21.15.3:0/2831566822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c011830 con 0x7f67800ce240 2024-09-13T16:51:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f677fe00700 1 -- 172.21.15.3:0/2831566822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c011990 con 0x7f67800ce240 2024-09-13T16:51:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.356+0000 7f677fe00700 1 -- 172.21.15.3:0/2831566822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c011d20 con 0x7f67800ce240 2024-09-13T16:51:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.360+0000 7f6786e00700 1 -- 172.21.15.3:0/2831566822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67800ce240 msgr2=0x7f67800b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.360+0000 7f6786e00700 1 --2- 172.21.15.3:0/2831566822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67800ce240 0x7f67800b3aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f676c009850 tx=0x7f676c009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.360+0000 7f6786e00700 1 -- 172.21.15.3:0/2831566822 shutdown_connections 2024-09-13T16:51:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.360+0000 7f6786e00700 1 --2- 172.21.15.3:0/2831566822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67800ce240 0x7f67800b3aa0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.360+0000 7f6786e00700 1 --2- 172.21.15.3:0/2831566822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678015ce20 0x7f67800cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.360+0000 7f6786e00700 1 -- 172.21.15.3:0/2831566822 >> 172.21.15.3:0/2831566822 conn(0x7f67800d2f10 msgr2=0x7f67800d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.360+0000 7f6786e00700 1 -- 172.21.15.3:0/2831566822 shutdown_connections 2024-09-13T16:51:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.360+0000 7f6786e00700 1 -- 172.21.15.3:0/2831566822 wait complete. 2024-09-13T16:51:43.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786e00700 1 Processor -- start 2024-09-13T16:51:43.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786e00700 1 -- start start 2024-09-13T16:51:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67800b3fe0 0x7f6780155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67800ce240 0x7f67800cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678015ce20 0x7f6780157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67800cc810 con 0x7f678015ce20 2024-09-13T16:51:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67800ccaf0 con 0x7f67800b3fe0 2024-09-13T16:51:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67800ccdd0 con 0x7f67800ce240 2024-09-13T16:51:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678015ce20 0x7f6780157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6785000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67800ce240 0x7f67800cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6785a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67800b3fe0 0x7f6780155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678015ce20 0x7f6780157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40710/0 (socket says 172.21.15.3:40710) 2024-09-13T16:51:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.364+0000 7f6786400700 1 -- 172.21.15.3:0/3360466264 learned_addr learned my addr 172.21.15.3:0/3360466264 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786400700 1 -- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67800ce240 msgr2=0x7f67800cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786400700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67800ce240 0x7f67800cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786400700 1 -- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67800b3fe0 msgr2=0x7f6780155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786400700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67800b3fe0 0x7f6780155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786400700 1 -- 172.21.15.3:0/3360466264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0094b0 con 0x7f678015ce20 2024-09-13T16:51:43.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786400700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678015ce20 0x7f6780157960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f677800cb10 tx=0x7f677800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:43.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f677d600700 1 -- 172.21.15.3:0/3360466264 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778007480 con 0x7f678015ce20 2024-09-13T16:51:43.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f677d600700 1 -- 172.21.15.3:0/3360466264 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67780075e0 con 0x7f678015ce20 2024-09-13T16:51:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f677d600700 1 -- 172.21.15.3:0/3360466264 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778007950 con 0x7f678015ce20 2024-09-13T16:51:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67800cd0b0 con 0x7f678015ce20 2024-09-13T16:51:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67800cc450 con 0x7f678015ce20 2024-09-13T16:51:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.368+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67801f3c20 con 0x7f678015ce20 2024-09-13T16:51:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.372+0000 7f677d600700 1 -- 172.21.15.3:0/3360466264 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f677801b3b0 con 0x7f678015ce20 2024-09-13T16:51:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.372+0000 7f677d600700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6760066730 0x7f6760068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.372+0000 7f677d600700 1 -- 172.21.15.3:0/3360466264 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6778083dd0 con 0x7f678015ce20 2024-09-13T16:51:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.376+0000 7f6785a00700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6760066730 0x7f6760068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.376+0000 7f6785a00700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6760066730 0x7f6760068bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f67801f7110 tx=0x7f6774008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.380+0000 7f677d600700 1 -- 172.21.15.3:0/3360466264 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f67801f3c20 con 0x7f678015ce20 2024-09-13T16:51:43.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.568+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67801f3c20 con 0x7f678015ce20 2024-09-13T16:51:43.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.572+0000 7f677d600700 1 -- 172.21.15.3:0/3360466264 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67801f3c20 con 0x7f678015ce20 2024-09-13T16:51:43.576 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6760066730 msgr2=0x7f6760068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6760066730 0x7f6760068bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f67801f7110 tx=0x7f6774008040 comp rx=0 tx=0).stop 2024-09-13T16:51:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678015ce20 msgr2=0x7f6780157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678015ce20 0x7f6780157960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f677800cb10 tx=0x7f677800cb40 comp rx=0 tx=0).stop 2024-09-13T16:51:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 shutdown_connections 2024-09-13T16:51:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6760066730 0x7f6760068bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678015ce20 0x7f6780157960 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67800ce240 0x7f67800cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 --2- 172.21.15.3:0/3360466264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67800b3fe0 0x7f6780155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 >> 172.21.15.3:0/3360466264 conn(0x7f67800d2f10 msgr2=0x7f67800b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 shutdown_connections 2024-09-13T16:51:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.576+0000 7f6786e00700 1 -- 172.21.15.3:0/3360466264 wait complete. 2024-09-13T16:51:43.605 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.772+0000 7f1840200700 1 Processor -- start 2024-09-13T16:51:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.772+0000 7f1840200700 1 -- start start 2024-09-13T16:51:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.772+0000 7f1840200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1838071690 0x7f183806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.772+0000 7f1840200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18380b3be0 0x7f183806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.772+0000 7f1840200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380b1f30 con 0x7f18380b3800 2024-09-13T16:51:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.772+0000 7f1840200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380aca50 con 0x7f18380b3be0 2024-09-13T16:51:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.772+0000 7f1840200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380723e0 con 0x7f1838071690 2024-09-13T16:51:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183e200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18380b3be0 0x7f183806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183e200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18380b3be0 0x7f183806d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38828/0 (socket says 172.21.15.3:38828) 2024-09-13T16:51:43.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183e200700 1 -- 172.21.15.3:0/3557549225 learned_addr learned my addr 172.21.15.3:0/3557549225 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:43.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183ce00700 1 --2- 172.21.15.3:0/3557549225 >> v2:172.21.15.157:3300/0 conn(0x7f1838071690 0x7f183806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183ce00700 1 -- 172.21.15.3:0/3557549225 >> v2:172.21.15.3:3300/0 conn(0x7f18380b3be0 msgr2=0x7f183806d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:43.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183ce00700 1 --2- 172.21.15.3:0/3557549225 >> v2:172.21.15.3:3300/0 conn(0x7f18380b3be0 0x7f183806d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183ce00700 1 -- 172.21.15.3:0/3557549225 >> v1:172.21.15.157:6789/0 conn(0x7f18380b3800 legacy=0x7f1838070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183ce00700 1 -- 172.21.15.3:0/3557549225 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1838071b40 con 0x7f1838071690 2024-09-13T16:51:43.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183e200700 1 --2- 172.21.15.3:0/3557549225 >> v2:172.21.15.3:3300/0 conn(0x7f18380b3be0 0x7f183806d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:43.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f183ce00700 1 --2- 172.21.15.3:0/3557549225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1838071690 0x7f183806d3b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f183815e500 tx=0x7f1824009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a7a836038e0eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:43.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f1837e00700 1 -- 172.21.15.3:0/3557549225 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824004230 con 0x7f1838071690 2024-09-13T16:51:43.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f1837e00700 1 -- 172.21.15.3:0/3557549225 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1824004390 con 0x7f1838071690 2024-09-13T16:51:43.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.776+0000 7f1837e00700 1 -- 172.21.15.3:0/3557549225 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240036c0 con 0x7f1838071690 2024-09-13T16:51:43.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.780+0000 7f1840200700 1 -- 172.21.15.3:0/3557549225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1838071690 msgr2=0x7f183806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.780+0000 7f1840200700 1 --2- 172.21.15.3:0/3557549225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1838071690 0x7f183806d3b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f183815e500 tx=0x7f1824009b30 comp rx=0 tx=0).stop 2024-09-13T16:51:43.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.780+0000 7f1840200700 1 -- 172.21.15.3:0/3557549225 shutdown_connections 2024-09-13T16:51:43.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.780+0000 7f1840200700 1 --2- 172.21.15.3:0/3557549225 >> v2:172.21.15.3:3300/0 conn(0x7f18380b3be0 0x7f183806d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.780+0000 7f1840200700 1 --2- 172.21.15.3:0/3557549225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1838071690 0x7f183806d3b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.780+0000 7f1840200700 1 -- 172.21.15.3:0/3557549225 >> 172.21.15.3:0/3557549225 conn(0x7f183806b280 msgr2=0x7f183806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:43.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.780+0000 7f1840200700 1 -- 172.21.15.3:0/3557549225 shutdown_connections 2024-09-13T16:51:43.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.780+0000 7f1840200700 1 -- 172.21.15.3:0/3557549225 wait complete. 2024-09-13T16:51:43.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f1840200700 1 Processor -- start 2024-09-13T16:51:43.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f1840200700 1 -- start start 2024-09-13T16:51:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f1840200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1838071690 0x7f183814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f1840200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18380b3800 0x7f183815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f1840200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18380b3be0 0x7f183815cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f1840200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183814acd0 con 0x7f18380b3be0 2024-09-13T16:51:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f1840200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381e60d0 con 0x7f18380b3800 2024-09-13T16:51:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f1840200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18381e6560 con 0x7f1838071690 2024-09-13T16:51:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f183e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18380b3be0 0x7f183815cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f183e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18380b3be0 0x7f183815cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38838/0 (socket says 172.21.15.3:38838) 2024-09-13T16:51:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f183e200700 1 -- 172.21.15.3:0/923385050 learned_addr learned my addr 172.21.15.3:0/923385050 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f183ce00700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18380b3800 0x7f183815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.784+0000 7f183d800700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1838071690 0x7f183814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f183ce00700 1 -- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1838071690 msgr2=0x7f183814a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f183ce00700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1838071690 0x7f183814a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f183ce00700 1 -- 172.21.15.3:0/923385050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18380b3be0 msgr2=0x7f183815cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f183ce00700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18380b3be0 0x7f183815cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f183ce00700 1 -- 172.21.15.3:0/923385050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830009b30 con 0x7f18380b3800 2024-09-13T16:51:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f183e200700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18380b3be0 0x7f183815cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f183ce00700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18380b3800 0x7f183815c7c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1824015000 tx=0x7f1824004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f1835600700 1 -- 172.21.15.3:0/923385050 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240045d0 con 0x7f18380b3800 2024-09-13T16:51:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f1835600700 1 -- 172.21.15.3:0/923385050 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182401f070 con 0x7f18380b3800 2024-09-13T16:51:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f1835600700 1 -- 172.21.15.3:0/923385050 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824011870 con 0x7f18380b3800 2024-09-13T16:51:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18240094b0 con 0x7f18380b3800 2024-09-13T16:51:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18381e57a0 con 0x7f18380b3800 2024-09-13T16:51:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.788+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18381e6240 con 0x7f18380b3800 2024-09-13T16:51:43.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.792+0000 7f1835600700 1 -- 172.21.15.3:0/923385050 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18240119d0 con 0x7f18380b3800 2024-09-13T16:51:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.792+0000 7f1835600700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f18180667b0 0x7f1818068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.792+0000 7f1835600700 1 -- 172.21.15.3:0/923385050 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1824021070 con 0x7f18380b3800 2024-09-13T16:51:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.796+0000 7f183d800700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f18180667b0 0x7f1818068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.796+0000 7f183d800700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f18180667b0 0x7f1818068c70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f182c007fd0 tx=0x7f182c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.804+0000 7f1835600700 1 -- 172.21.15.3:0/923385050 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1824052c10 con 0x7f18380b3800 2024-09-13T16:51:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.992+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f183806b8f0 con 0x7f18380b3800 2024-09-13T16:51:43.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.992+0000 7f1835600700 1 -- 172.21.15.3:0/923385050 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1824056230 con 0x7f18380b3800 2024-09-13T16:51:43.998 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f18180667b0 msgr2=0x7f1818068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f18180667b0 0x7f1818068c70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f182c007fd0 tx=0x7f182c009040 comp rx=0 tx=0).stop 2024-09-13T16:51:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18380b3800 msgr2=0x7f183815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18380b3800 0x7f183815c7c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1824015000 tx=0x7f1824004770 comp rx=0 tx=0).stop 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 shutdown_connections 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f18180667b0 0x7f1818068c70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18380b3be0 0x7f183815cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18380b3800 0x7f183815c7c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 --2- 172.21.15.3:0/923385050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1838071690 0x7f183814a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 >> 172.21.15.3:0/923385050 conn(0x7f183806b280 msgr2=0x7f183815f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 shutdown_connections 2024-09-13T16:51:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:43.996+0000 7f1840200700 1 -- 172.21.15.3:0/923385050 wait complete. 2024-09-13T16:51:44.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:44.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.204+0000 7f132ec00700 1 Processor -- start 2024-09-13T16:51:44.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.204+0000 7f132ec00700 1 -- start start 2024-09-13T16:51:44.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f132ec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13280ccf40 0x7f13280ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f132ec00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280b1f30 con 0x7f1328158740 2024-09-13T16:51:44.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f132ec00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132815b710 con 0x7f132815b1c0 2024-09-13T16:51:44.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f132ec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13280b3960 con 0x7f13280ccf40 2024-09-13T16:51:44.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f1327e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1328158740 0x7f13280b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47278/0 (socket says 172.21.15.3:47278) 2024-09-13T16:51:44.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f1327e00700 1 -- 172.21.15.3:0/2396989348 learned_addr learned my addr 172.21.15.3:0/2396989348 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:44.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f132cc00700 1 --2- 172.21.15.3:0/2396989348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13280ccf40 0x7f13280ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f1326a00700 1 -- 172.21.15.3:0/2396989348 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114183020 0 0) 0x7f13280b1f30 con 0x7f1328158740 2024-09-13T16:51:44.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f1326a00700 1 -- 172.21.15.3:0/2396989348 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f130c0035d0 con 0x7f1328158740 2024-09-13T16:51:44.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f1326a00700 1 -- 172.21.15.3:0/2396989348 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196803162 0 0) 0x7f132815b710 con 0x7f132815b1c0 2024-09-13T16:51:44.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f1326a00700 1 -- 172.21.15.3:0/2396989348 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13280b1f30 con 0x7f132815b1c0 2024-09-13T16:51:44.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f132cc00700 1 -- 172.21.15.3:0/2396989348 >> v1:172.21.15.157:6789/0 conn(0x7f132815b1c0 legacy=0x7f13280cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f1326a00700 1 -- 172.21.15.3:0/2396989348 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 61177495 0 0) 0x7f130c0035d0 con 0x7f1328158740 2024-09-13T16:51:44.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f132cc00700 1 -- 172.21.15.3:0/2396989348 >> v1:172.21.15.3:6789/0 conn(0x7f1328158740 legacy=0x7f13280b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.208+0000 7f132cc00700 1 -- 172.21.15.3:0/2396989348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132815b9f0 con 0x7f13280ccf40 2024-09-13T16:51:44.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f132cc00700 1 --2- 172.21.15.3:0/2396989348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13280ccf40 0x7f13280ce6e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1320009dc0 tx=0x7f132000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=640faf7173517a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:44.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f1326a00700 1 -- 172.21.15.3:0/2396989348 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000e070 con 0x7f13280ccf40 2024-09-13T16:51:44.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f1326a00700 1 -- 172.21.15.3:0/2396989348 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1320003a40 con 0x7f13280ccf40 2024-09-13T16:51:44.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f1326a00700 1 -- 172.21.15.3:0/2396989348 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318001670 con 0x7f13280ccf40 2024-09-13T16:51:44.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f132ec00700 1 -- 172.21.15.3:0/2396989348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13280ccf40 msgr2=0x7f13280ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f132ec00700 1 --2- 172.21.15.3:0/2396989348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13280ccf40 0x7f13280ce6e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1320009dc0 tx=0x7f132000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:44.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f132ec00700 1 -- 172.21.15.3:0/2396989348 shutdown_connections 2024-09-13T16:51:44.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f132ec00700 1 --2- 172.21.15.3:0/2396989348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13280ccf40 0x7f13280ce6e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f132ec00700 1 -- 172.21.15.3:0/2396989348 >> 172.21.15.3:0/2396989348 conn(0x7f132806b280 msgr2=0x7f132806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:44.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f132ec00700 1 -- 172.21.15.3:0/2396989348 shutdown_connections 2024-09-13T16:51:44.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.212+0000 7f132ec00700 1 -- 172.21.15.3:0/2396989348 wait complete. 2024-09-13T16:51:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132ec00700 1 Processor -- start 2024-09-13T16:51:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132ec00700 1 -- start start 2024-09-13T16:51:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280ccf40 0x7f132814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132ec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1328158740 0x7f1328152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132ec00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f132815b1c0 0x7f132815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132ec00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1328154ee0 con 0x7f132815b1c0 2024-09-13T16:51:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132ec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132814df20 con 0x7f13280ccf40 2024-09-13T16:51:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132ec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1328155300 con 0x7f1328158740 2024-09-13T16:51:44.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.216+0000 7f132cc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f132815b1c0 0x7f132815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f1327e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280ccf40 0x7f132814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f1327e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280ccf40 0x7f132814d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:41674/0 (socket says 172.21.15.3:41674) 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f1327e00700 1 -- 172.21.15.3:0/605470306 learned_addr learned my addr 172.21.15.3:0/605470306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f1327400700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1328158740 0x7f1328152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132cc00700 1 -- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1328158740 msgr2=0x7f1328152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132cc00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1328158740 0x7f1328152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132cc00700 1 -- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280ccf40 msgr2=0x7f132814d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132cc00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280ccf40 0x7f132814d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132cc00700 1 -- 172.21.15.3:0/605470306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320009a70 con 0x7f132815b1c0 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f1327e00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280ccf40 0x7f132814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132cc00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f132815b1c0 0x7f132815d5d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1320009fd0 tx=0x7f1320004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f131fe00700 1 -- 172.21.15.3:0/605470306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13200042c0 con 0x7f132815b1c0 2024-09-13T16:51:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f131fe00700 1 -- 172.21.15.3:0/605470306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1320004440 con 0x7f132815b1c0 2024-09-13T16:51:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f131fe00700 1 -- 172.21.15.3:0/605470306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13200047f0 con 0x7f132815b1c0 2024-09-13T16:51:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13281542f0 con 0x7f132815b1c0 2024-09-13T16:51:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1328154820 con 0x7f132815b1c0 2024-09-13T16:51:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.220+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1328155020 con 0x7f132815b1c0 2024-09-13T16:51:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.224+0000 7f131fe00700 1 -- 172.21.15.3:0/605470306 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f132001c460 con 0x7f132815b1c0 2024-09-13T16:51:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.224+0000 7f131fe00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.224+0000 7f131fe00700 1 -- 172.21.15.3:0/605470306 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1320085af0 con 0x7f132815b1c0 2024-09-13T16:51:44.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.228+0000 7f1327e00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.228+0000 7f1327e00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f13180015a0 tx=0x7f13180014f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:44.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.232+0000 7f131fe00700 1 -- 172.21.15.3:0/605470306 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1328155020 con 0x7f132815b1c0 2024-09-13T16:51:44.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.412+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1328155020 con 0x7f132815b1c0 2024-09-13T16:51:44.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.412+0000 7f131fe00700 1 -- 172.21.15.3:0/605470306 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1328155020 con 0x7f132815b1c0 2024-09-13T16:51:44.420 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 msgr2=0x7f1308068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f13180015a0 tx=0x7f13180014f0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f132815b1c0 msgr2=0x7f132815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f132815b1c0 0x7f132815d5d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1320009fd0 tx=0x7f1320004af0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 shutdown_connections 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1308066730 0x7f1308068bf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f132815b1c0 0x7f132815d5d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1328158740 0x7f1328152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 --2- 172.21.15.3:0/605470306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280ccf40 0x7f132814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 >> 172.21.15.3:0/605470306 conn(0x7f132806b280 msgr2=0x7f13280b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 shutdown_connections 2024-09-13T16:51:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.416+0000 7f132ec00700 1 -- 172.21.15.3:0/605470306 wait complete. 2024-09-13T16:51:44.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:51:44.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.608+0000 7f415f000700 1 Processor -- start 2024-09-13T16:51:44.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.608+0000 7f415f000700 1 -- start start 2024-09-13T16:51:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.608+0000 7f415f000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f415806ccf0 0x7f4158165820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.608+0000 7f415f000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41580b4100 con 0x7f41581443b0 2024-09-13T16:51:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.608+0000 7f415f000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41580ce8e0 con 0x7f4158144790 2024-09-13T16:51:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.608+0000 7f415f000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41580ccd30 con 0x7f415806ccf0 2024-09-13T16:51:44.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415dc00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f41581443b0 0x7f41580b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47284/0 (socket says 172.21.15.3:47284) 2024-09-13T16:51:44.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415dc00700 1 -- 172.21.15.3:0/1464336046 learned_addr learned my addr 172.21.15.3:0/1464336046 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:44.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415e600700 1 --2- 172.21.15.3:0/1464336046 >> v2:172.21.15.157:3300/0 conn(0x7f415806ccf0 0x7f4158165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f4157e00700 1 -- 172.21.15.3:0/1464336046 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589267975 0 0) 0x7f41580b4100 con 0x7f41581443b0 2024-09-13T16:51:44.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f4157e00700 1 -- 172.21.15.3:0/1464336046 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41400035d0 con 0x7f41581443b0 2024-09-13T16:51:44.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f4157e00700 1 -- 172.21.15.3:0/1464336046 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890370724 0 0) 0x7f41580ce8e0 con 0x7f4158144790 2024-09-13T16:51:44.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f4157e00700 1 -- 172.21.15.3:0/1464336046 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41580b4100 con 0x7f4158144790 2024-09-13T16:51:44.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415e600700 1 -- 172.21.15.3:0/1464336046 >> v1:172.21.15.157:6789/0 conn(0x7f4158144790 legacy=0x7f415806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415e600700 1 -- 172.21.15.3:0/1464336046 >> v1:172.21.15.3:6789/0 conn(0x7f41581443b0 legacy=0x7f41580b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415e600700 1 -- 172.21.15.3:0/1464336046 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580cd010 con 0x7f415806ccf0 2024-09-13T16:51:44.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415e600700 1 --2- 172.21.15.3:0/1464336046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 0x7f4158165820 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4150009d90 tx=0x7f415000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d2c660c07413c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:44.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f4157e00700 1 -- 172.21.15.3:0/1464336046 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415000e070 con 0x7f415806ccf0 2024-09-13T16:51:44.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f4157e00700 1 -- 172.21.15.3:0/1464336046 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415000bb20 con 0x7f415806ccf0 2024-09-13T16:51:44.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f4157e00700 1 -- 172.21.15.3:0/1464336046 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4150012430 con 0x7f415806ccf0 2024-09-13T16:51:44.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415f000700 1 -- 172.21.15.3:0/1464336046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 msgr2=0x7f4158165820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.612+0000 7f415f000700 1 --2- 172.21.15.3:0/1464336046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 0x7f4158165820 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4150009d90 tx=0x7f415000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:44.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.616+0000 7f415f000700 1 -- 172.21.15.3:0/1464336046 shutdown_connections 2024-09-13T16:51:44.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.616+0000 7f415f000700 1 --2- 172.21.15.3:0/1464336046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 0x7f4158165820 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.616+0000 7f415f000700 1 -- 172.21.15.3:0/1464336046 >> 172.21.15.3:0/1464336046 conn(0x7f41580b0cf0 msgr2=0x7f41580b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:44.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.616+0000 7f415f000700 1 -- 172.21.15.3:0/1464336046 shutdown_connections 2024-09-13T16:51:44.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.616+0000 7f415f000700 1 -- 172.21.15.3:0/1464336046 wait complete. 2024-09-13T16:51:44.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.616+0000 7f415f000700 1 Processor -- start 2024-09-13T16:51:44.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.616+0000 7f415f000700 1 -- start start 2024-09-13T16:51:44.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 0x7f4158071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415f000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41581443b0 0x7f41581f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415f000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41580715b0 0x7f41581f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415f000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4158156660 con 0x7f41581443b0 2024-09-13T16:51:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41581f24f0 con 0x7f415806ccf0 2024-09-13T16:51:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415f000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415806ffd0 con 0x7f41580715b0 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41581443b0 0x7f41581f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41581443b0 0x7f41581f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38842/0 (socket says 172.21.15.3:38842) 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 -- 172.21.15.3:0/2756462288 learned_addr learned my addr 172.21.15.3:0/2756462288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415dc00700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 0x7f4158071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415e600700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41580715b0 0x7f41581f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 -- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41580715b0 msgr2=0x7f41581f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41580715b0 0x7f41581f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 -- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 msgr2=0x7f4158071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 0x7f4158071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 -- 172.21.15.3:0/2756462288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4150009a70 con 0x7f41581443b0 2024-09-13T16:51:44.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415d200700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41581443b0 0x7f41581f5c00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f414400c490 tx=0x7f414400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:44.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f4155600700 1 -- 172.21.15.3:0/2756462288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400cea0 con 0x7f41581443b0 2024-09-13T16:51:44.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f4155600700 1 -- 172.21.15.3:0/2756462288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41440052f0 con 0x7f41581443b0 2024-09-13T16:51:44.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f4155600700 1 -- 172.21.15.3:0/2756462288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440036a0 con 0x7f41581443b0 2024-09-13T16:51:44.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41580702b0 con 0x7f41581443b0 2024-09-13T16:51:44.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.620+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4158070870 con 0x7f41581443b0 2024-09-13T16:51:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.624+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41580b39f0 con 0x7f41581443b0 2024-09-13T16:51:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.624+0000 7f4155600700 1 -- 172.21.15.3:0/2756462288 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4144003800 con 0x7f41581443b0 2024-09-13T16:51:44.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.628+0000 7f4155600700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4138066730 0x7f4138068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:44.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.628+0000 7f4155600700 1 -- 172.21.15.3:0/2756462288 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4144083910 con 0x7f41581443b0 2024-09-13T16:51:44.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.628+0000 7f415dc00700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4138066730 0x7f4138068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:44.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.628+0000 7f415dc00700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4138066730 0x7f4138068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f414c007fd0 tx=0x7f414c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:44.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.632+0000 7f4155600700 1 -- 172.21.15.3:0/2756462288 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f41440508b0 con 0x7f41581443b0 2024-09-13T16:51:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.836+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f41580b0530 con 0x7f41581443b0 2024-09-13T16:51:44.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.836+0000 7f4155600700 1 -- 172.21.15.3:0/2756462288 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4144053ed0 con 0x7f41581443b0 2024-09-13T16:51:44.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4138066730 msgr2=0x7f4138068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4138066730 0x7f4138068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f414c007fd0 tx=0x7f414c007de0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41581443b0 msgr2=0x7f41581f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41581443b0 0x7f41581f5c00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f414400c490 tx=0x7f414400c7a0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 shutdown_connections 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4138066730 0x7f4138068bf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41580715b0 0x7f41581f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41581443b0 0x7f41581f5c00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 --2- 172.21.15.3:0/2756462288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415806ccf0 0x7f4158071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 >> 172.21.15.3:0/2756462288 conn(0x7f41580b0cf0 msgr2=0x7f41580cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 shutdown_connections 2024-09-13T16:51:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:44.840+0000 7f415f000700 1 -- 172.21.15.3:0/2756462288 wait complete. 2024-09-13T16:51:44.847 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-13T16:51:44.869 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-13T16:51:44.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:44.870 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-13T16:51:44.919 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:51:44.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:51:44.919 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-13T16:51:44.982 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T16:51:44.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.156+0000 7f47b6000700 1 Processor -- start 2024-09-13T16:51:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.156+0000 7f47b6000700 1 -- start start 2024-09-13T16:51:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b6000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47b015cdd0 0x7f47b00d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b6000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b00d1510 con 0x7f47b00d25b0 2024-09-13T16:51:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b6000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b00d17f0 con 0x7f47b015d1b0 2024-09-13T16:51:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b6000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00d1ad0 con 0x7f47b015cdd0 2024-09-13T16:51:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b5600700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f47b00d25b0 0x7f47b00cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47300/0 (socket says 172.21.15.3:47300) 2024-09-13T16:51:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b5600700 1 -- 172.21.15.3:0/4059454974 learned_addr learned my addr 172.21.15.3:0/4059454974 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b4c00700 1 --2- 172.21.15.3:0/4059454974 >> v2:172.21.15.157:3300/0 conn(0x7f47b015cdd0 0x7f47b00d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367454701 0 0) 0x7f47b00d1510 con 0x7f47b00d25b0 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47940035d0 con 0x7f47b00d25b0 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400704028 0 0) 0x7f47b00d17f0 con 0x7f47b015d1b0 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b00d1510 con 0x7f47b015d1b0 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3863551612 0 0) 0x7f47940035d0 con 0x7f47b00d25b0 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 >> v2:172.21.15.157:3300/0 conn(0x7f47b015cdd0 msgr2=0x7f47b00d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 --2- 172.21.15.3:0/4059454974 >> v2:172.21.15.157:3300/0 conn(0x7f47b015cdd0 0x7f47b00d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 >> v1:172.21.15.157:6789/0 conn(0x7f47b015d1b0 legacy=0x7f47b00b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00d1db0 con 0x7f47b00d25b0 2024-09-13T16:51:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f47a8001670 con 0x7f47b00d25b0 2024-09-13T16:51:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 >> v1:172.21.15.3:6789/0 conn(0x7f47b00d25b0 legacy=0x7f47b00cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794003bb0 0x7f4794003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4794004870 0x7f4794004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4794005230 0x7f479400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00d1ad0 con 0x7f4794004870 2024-09-13T16:51:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479400c0b0 con 0x7f4794005230 2024-09-13T16:51:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479400c270 con 0x7f4794003bb0 2024-09-13T16:51:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47a7e00700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4794004870 0x7f4794004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b4c00700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794003bb0 0x7f4794003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.160+0000 7f47b5600700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4794005230 0x7f479400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47a7e00700 1 -- 172.21.15.3:0/4059454974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794003bb0 msgr2=0x7f4794003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47a7e00700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794003bb0 0x7f4794003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47a7e00700 1 -- 172.21.15.3:0/4059454974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4794005230 msgr2=0x7f479400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47a7e00700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4794005230 0x7f479400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47b5600700 1 -- 172.21.15.3:0/4059454974 reap_dead start 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47a7e00700 1 -- 172.21.15.3:0/4059454974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47940035d0 con 0x7f4794004870 2024-09-13T16:51:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47a7e00700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4794004870 0x7f4794004cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4798001960 tx=0x7f47940058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d5a009c8a5bd0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798001450 con 0x7f4794004870 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4798006aa0 con 0x7f4794004870 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47afe00700 1 -- 172.21.15.3:0/4059454974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798006e50 con 0x7f4794004870 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47b6000700 1 -- 172.21.15.3:0/4059454974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4794004870 msgr2=0x7f4794004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47b6000700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4794004870 0x7f4794004cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4798001960 tx=0x7f47940058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47b6000700 1 -- 172.21.15.3:0/4059454974 shutdown_connections 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47b6000700 1 --2- 172.21.15.3:0/4059454974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4794004870 0x7f4794004cf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47b6000700 1 -- 172.21.15.3:0/4059454974 >> 172.21.15.3:0/4059454974 conn(0x7f47b00d3190 msgr2=0x7f47b00d35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47b6000700 1 -- 172.21.15.3:0/4059454974 shutdown_connections 2024-09-13T16:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.164+0000 7f47b6000700 1 -- 172.21.15.3:0/4059454974 wait complete. 2024-09-13T16:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b6000700 1 Processor -- start 2024-09-13T16:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b6000700 1 -- start start 2024-09-13T16:51:45.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794004870 0x7f47b01602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00c66d0 0x7f47b00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b6000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47b014fee0 0x7f47b0150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b6000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0150890 con 0x7f47b014fee0 2024-09-13T16:51:45.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b6000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00c6bb0 con 0x7f47b00c66d0 2024-09-13T16:51:45.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b6000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00c6e90 con 0x7f4794004870 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47b014fee0 0x7f47b0150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47b014fee0 0x7f47b0150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38860/0 (socket says 172.21.15.3:38860) 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b5600700 1 -- 172.21.15.3:0/2788903237 learned_addr learned my addr 172.21.15.3:0/2788903237 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47b4c00700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794004870 0x7f47b01602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.168+0000 7f47a7e00700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00c66d0 0x7f47b00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b5600700 1 -- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794004870 msgr2=0x7f47b01602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b5600700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794004870 0x7f47b01602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b5600700 1 -- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00c66d0 msgr2=0x7f47b00cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b5600700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00c66d0 0x7f47b00cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b5600700 1 -- 172.21.15.3:0/2788903237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00c7170 con 0x7f47b014fee0 2024-09-13T16:51:45.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47a7e00700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00c66d0 0x7f47b00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:45.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b5600700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47b014fee0 0x7f47b0150350 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f47b00c66a0 tx=0x7f47a800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:45.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47ad600700 1 -- 172.21.15.3:0/2788903237 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8014480 con 0x7f47b014fee0 2024-09-13T16:51:45.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47ad600700 1 -- 172.21.15.3:0/2788903237 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a80145e0 con 0x7f47b014fee0 2024-09-13T16:51:45.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47ad600700 1 -- 172.21.15.3:0/2788903237 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8014990 con 0x7f47b014fee0 2024-09-13T16:51:45.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b00c7450 con 0x7f47b014fee0 2024-09-13T16:51:45.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b015fef0 con 0x7f47b014fee0 2024-09-13T16:51:45.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.172+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b0067310 con 0x7f47b014fee0 2024-09-13T16:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.176+0000 7f47ad600700 1 -- 172.21.15.3:0/2788903237 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47a80114f0 con 0x7f47b014fee0 2024-09-13T16:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.176+0000 7f47ad600700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f478c066760 0x7f478c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.176+0000 7f47ad600700 1 -- 172.21.15.3:0/2788903237 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f47a8083ae0 con 0x7f47b014fee0 2024-09-13T16:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.176+0000 7f47b4c00700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f478c066760 0x7f478c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.180+0000 7f47b4c00700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f478c066760 0x7f478c068c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f47b0069780 tx=0x7f47a000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.184+0000 7f47ad600700 1 -- 172.21.15.3:0/2788903237 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f47a8050a80 con 0x7f47b014fee0 2024-09-13T16:51:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.412+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47b00d17f0 con 0x7f47b014fee0 2024-09-13T16:51:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.412+0000 7f47ad600700 1 -- 172.21.15.3:0/2788903237 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f47a8019020 con 0x7f47b014fee0 2024-09-13T16:51:45.418 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:45.418 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:55.700655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:51:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f478c066760 msgr2=0x7f478c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f478c066760 0x7f478c068c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f47b0069780 tx=0x7f47a000b040 comp rx=0 tx=0).stop 2024-09-13T16:51:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47b014fee0 msgr2=0x7f47b0150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47b014fee0 0x7f47b0150350 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f47b00c66a0 tx=0x7f47a800e830 comp rx=0 tx=0).stop 2024-09-13T16:51:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 shutdown_connections 2024-09-13T16:51:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f478c066760 0x7f478c068c20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47b014fee0 0x7f47b0150350 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00c66d0 0x7f47b00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 --2- 172.21.15.3:0/2788903237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4794004870 0x7f47b01602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 >> 172.21.15.3:0/2788903237 conn(0x7f47b00d3190 msgr2=0x7f47b00cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 shutdown_connections 2024-09-13T16:51:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.416+0000 7f47b6000700 1 -- 172.21.15.3:0/2788903237 wait complete. 2024-09-13T16:51:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-13T16:51:45.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:45.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.612+0000 7fa26de00700 1 Processor -- start 2024-09-13T16:51:45.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.612+0000 7fa26de00700 1 -- start start 2024-09-13T16:51:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa26de00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2681443b0 0x7fa2680b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa26de00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa268144790 0x7fa26806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa26de00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26806cf70 con 0x7fa26806cb20 2024-09-13T16:51:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa26de00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2680ce8e0 con 0x7fa268144790 2024-09-13T16:51:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa26de00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2680ccd30 con 0x7fa2681443b0 2024-09-13T16:51:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa25fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa268144790 0x7fa26806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa25fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa268144790 0x7fa26806c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38874/0 (socket says 172.21.15.3:38874) 2024-09-13T16:51:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa25fe00700 1 -- 172.21.15.3:0/1578013480 learned_addr learned my addr 172.21.15.3:0/1578013480 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa26ca00700 1 --2- 172.21.15.3:0/1578013480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2681443b0 0x7fa2680b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa25fe00700 1 -- 172.21.15.3:0/1578013480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2681443b0 msgr2=0x7fa2680b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa25fe00700 1 --2- 172.21.15.3:0/1578013480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2681443b0 0x7fa2680b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa25fe00700 1 -- 172.21.15.3:0/1578013480 >> v1:172.21.15.157:6789/0 conn(0x7fa26806cb20 legacy=0x7fa26806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa25fe00700 1 -- 172.21.15.3:0/1578013480 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2680cd010 con 0x7fa268144790 2024-09-13T16:51:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.616+0000 7fa25fe00700 1 --2- 172.21.15.3:0/1578013480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268144790 0x7fa26806c5e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa250009910 tx=0x7fa250009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fb45060ef7c3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa267e00700 1 -- 172.21.15.3:0/1578013480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011830 con 0x7fa268144790 2024-09-13T16:51:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa267e00700 1 -- 172.21.15.3:0/1578013480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa250011990 con 0x7fa268144790 2024-09-13T16:51:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa267e00700 1 -- 172.21.15.3:0/1578013480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011d40 con 0x7fa268144790 2024-09-13T16:51:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa26de00700 1 -- 172.21.15.3:0/1578013480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268144790 msgr2=0x7fa26806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa26de00700 1 --2- 172.21.15.3:0/1578013480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268144790 0x7fa26806c5e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa250009910 tx=0x7fa250009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa26de00700 1 -- 172.21.15.3:0/1578013480 shutdown_connections 2024-09-13T16:51:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa26de00700 1 --2- 172.21.15.3:0/1578013480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268144790 0x7fa26806c5e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa26de00700 1 --2- 172.21.15.3:0/1578013480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2681443b0 0x7fa2680b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa26de00700 1 -- 172.21.15.3:0/1578013480 >> 172.21.15.3:0/1578013480 conn(0x7fa2680b0cf0 msgr2=0x7fa2680b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa26de00700 1 -- 172.21.15.3:0/1578013480 shutdown_connections 2024-09-13T16:51:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.620+0000 7fa26de00700 1 -- 172.21.15.3:0/1578013480 wait complete. 2024-09-13T16:51:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26de00700 1 Processor -- start 2024-09-13T16:51:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26de00700 1 -- start start 2024-09-13T16:51:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26806cb20 0x7fa2681fa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2681443b0 0x7fa26815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26de00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2680c5120 0x7fa2680a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26de00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2680a97f0 con 0x7fa26806cb20 2024-09-13T16:51:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26de00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2680a8c20 con 0x7fa2681443b0 2024-09-13T16:51:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26de00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2681fab80 con 0x7fa2680c5120 2024-09-13T16:51:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26ca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26806cb20 0x7fa2681fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26ca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26806cb20 0x7fa2681fa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38878/0 (socket says 172.21.15.3:38878) 2024-09-13T16:51:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26ca00700 1 -- 172.21.15.3:0/2171301343 learned_addr learned my addr 172.21.15.3:0/2171301343 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26d400700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2680c5120 0x7fa2680a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa25fe00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2681443b0 0x7fa26815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26ca00700 1 -- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2680c5120 msgr2=0x7fa2680a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26ca00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2680c5120 0x7fa2680a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26ca00700 1 -- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2681443b0 msgr2=0x7fa26815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26ca00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2681443b0 0x7fa26815a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.624+0000 7fa26ca00700 1 -- 172.21.15.3:0/2171301343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa250009570 con 0x7fa26806cb20 2024-09-13T16:51:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.628+0000 7fa26ca00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26806cb20 0x7fa2681fa640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa25800efc0 tx=0x7fa25800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.628+0000 7fa265600700 1 -- 172.21.15.3:0/2171301343 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25800cd10 con 0x7fa26806cb20 2024-09-13T16:51:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.628+0000 7fa265600700 1 -- 172.21.15.3:0/2171301343 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25800f040 con 0x7fa26806cb20 2024-09-13T16:51:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.628+0000 7fa265600700 1 -- 172.21.15.3:0/2171301343 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2580146a0 con 0x7fa26806cb20 2024-09-13T16:51:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.628+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2680a8ec0 con 0x7fa26806cb20 2024-09-13T16:51:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.628+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2680a9480 con 0x7fa26806cb20 2024-09-13T16:51:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.628+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26806c5e0 con 0x7fa26806cb20 2024-09-13T16:51:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.628+0000 7fa265600700 1 -- 172.21.15.3:0/2171301343 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa258014800 con 0x7fa26806cb20 2024-09-13T16:51:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.632+0000 7fa265600700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa244066730 0x7fa244068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.632+0000 7fa265600700 1 -- 172.21.15.3:0/2171301343 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa258084d10 con 0x7fa26806cb20 2024-09-13T16:51:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.632+0000 7fa25fe00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa244066730 0x7fa244068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.632+0000 7fa25fe00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa244066730 0x7fa244068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa250000c00 tx=0x7fa250008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.636+0000 7fa265600700 1 -- 172.21.15.3:0/2171301343 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa258051d50 con 0x7fa26806cb20 2024-09-13T16:51:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.832+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2680ae290 con 0x7fa26806cb20 2024-09-13T16:51:45.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.832+0000 7fa265600700 1 -- 172.21.15.3:0/2171301343 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7fa258055370 con 0x7fa26806cb20 2024-09-13T16:51:45.838 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa244066730 msgr2=0x7fa244068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa244066730 0x7fa244068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa250000c00 tx=0x7fa250008030 comp rx=0 tx=0).stop 2024-09-13T16:51:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26806cb20 msgr2=0x7fa2681fa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26806cb20 0x7fa2681fa640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa25800efc0 tx=0x7fa25800c3a0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 shutdown_connections 2024-09-13T16:51:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa244066730 0x7fa244068bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2680c5120 0x7fa2680a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2681443b0 0x7fa26815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 --2- 172.21.15.3:0/2171301343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26806cb20 0x7fa2681fa640 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 >> 172.21.15.3:0/2171301343 conn(0x7fa2680b0cf0 msgr2=0x7fa2680ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 shutdown_connections 2024-09-13T16:51:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:45.836+0000 7fa26de00700 1 -- 172.21.15.3:0/2171301343 wait complete. 2024-09-13T16:51:45.866 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:50:28.793699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:50:29.563248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:50:32.649835+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:51:45.866 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.028+0000 7f8e33e00700 1 Processor -- start 2024-09-13T16:51:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.028+0000 7f8e33e00700 1 -- start start 2024-09-13T16:51:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e33e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8e341507b0 0x7f8e34159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e33e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e3406aaa0 0x7f8e3406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e33e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3406cba0 con 0x7f8e340b2ce0 2024-09-13T16:51:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e33e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3406e180 con 0x7f8e3406aaa0 2024-09-13T16:51:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e33e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3406ce80 con 0x7f8e341507b0 2024-09-13T16:51:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e33400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e3406aaa0 0x7f8e3406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e32a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8e341507b0 0x7f8e34159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e33400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e3406aaa0 0x7f8e3406dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38890/0 (socket says 172.21.15.3:38890) 2024-09-13T16:51:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.032+0000 7f8e33400700 1 -- 172.21.15.3:0/55740986 learned_addr learned my addr 172.21.15.3:0/55740986 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e32a00700 1 -- 172.21.15.3:0/55740986 >> v2:172.21.15.3:3300/0 conn(0x7f8e3406aaa0 msgr2=0x7f8e3406dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e32a00700 1 --2- 172.21.15.3:0/55740986 >> v2:172.21.15.3:3300/0 conn(0x7f8e3406aaa0 0x7f8e3406dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e32a00700 1 -- 172.21.15.3:0/55740986 >> v1:172.21.15.3:6789/0 conn(0x7f8e340b2ce0 legacy=0x7f8e3406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e32a00700 1 -- 172.21.15.3:0/55740986 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3406d160 con 0x7f8e341507b0 2024-09-13T16:51:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e32a00700 1 --2- 172.21.15.3:0/55740986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e341507b0 0x7f8e34159e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e340b1880 tx=0x7f8e24009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea3d45d110957bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e31600700 1 -- 172.21.15.3:0/55740986 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24011830 con 0x7f8e341507b0 2024-09-13T16:51:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e31600700 1 -- 172.21.15.3:0/55740986 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e24011990 con 0x7f8e341507b0 2024-09-13T16:51:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e31600700 1 -- 172.21.15.3:0/55740986 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24011d20 con 0x7f8e341507b0 2024-09-13T16:51:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e33e00700 1 -- 172.21.15.3:0/55740986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e341507b0 msgr2=0x7f8e34159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e33e00700 1 --2- 172.21.15.3:0/55740986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e341507b0 0x7f8e34159e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e340b1880 tx=0x7f8e24009b30 comp rx=0 tx=0).stop 2024-09-13T16:51:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e33e00700 1 -- 172.21.15.3:0/55740986 shutdown_connections 2024-09-13T16:51:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e33e00700 1 --2- 172.21.15.3:0/55740986 >> v2:172.21.15.3:3300/0 conn(0x7f8e3406aaa0 0x7f8e3406dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e33e00700 1 --2- 172.21.15.3:0/55740986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e341507b0 0x7f8e34159e20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e33e00700 1 -- 172.21.15.3:0/55740986 >> 172.21.15.3:0/55740986 conn(0x7f8e340af620 msgr2=0x7f8e340afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e33e00700 1 -- 172.21.15.3:0/55740986 shutdown_connections 2024-09-13T16:51:46.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.036+0000 7f8e33e00700 1 -- 172.21.15.3:0/55740986 wait complete. 2024-09-13T16:51:46.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.040+0000 7f8e33e00700 1 Processor -- start 2024-09-13T16:51:46.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 -- start start 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3406aaa0 0x7f8e3414cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e340b2ce0 0x7f8e34154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e341507b0 0x7f8e340b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e341557b0 con 0x7f8e340b2ce0 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e34154f10 con 0x7f8e3406aaa0 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e341551f0 con 0x7f8e341507b0 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e340b2ce0 0x7f8e34154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e340b2ce0 0x7f8e34154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38898/0 (socket says 172.21.15.3:38898) 2024-09-13T16:51:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 -- 172.21.15.3:0/2903223959 learned_addr learned my addr 172.21.15.3:0/2903223959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32a00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3406aaa0 0x7f8e3414cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33400700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e341507b0 0x7f8e340b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 -- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e341507b0 msgr2=0x7f8e340b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e341507b0 0x7f8e340b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 -- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3406aaa0 msgr2=0x7f8e3414cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3406aaa0 0x7f8e3414cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 -- 172.21.15.3:0/2903223959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e240094b0 con 0x7f8e340b2ce0 2024-09-13T16:51:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32a00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3406aaa0 0x7f8e3414cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e32000700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e340b2ce0 0x7f8e34154450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000c500 tx=0x7f8e2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e2aa00700 1 -- 172.21.15.3:0/2903223959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e200052d0 con 0x7f8e340b2ce0 2024-09-13T16:51:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e2aa00700 1 -- 172.21.15.3:0/2903223959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e20005430 con 0x7f8e340b2ce0 2024-09-13T16:51:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e341554d0 con 0x7f8e340b2ce0 2024-09-13T16:51:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e2aa00700 1 -- 172.21.15.3:0/2903223959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e200055a0 con 0x7f8e340b2ce0 2024-09-13T16:51:46.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.044+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3414da40 con 0x7f8e340b2ce0 2024-09-13T16:51:46.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.048+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e34158460 con 0x7f8e340b2ce0 2024-09-13T16:51:46.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.048+0000 7f8e2aa00700 1 -- 172.21.15.3:0/2903223959 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8e200038e0 con 0x7f8e340b2ce0 2024-09-13T16:51:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.052+0000 7f8e2aa00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e14066760 0x7f8e14068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.052+0000 7f8e2aa00700 1 -- 172.21.15.3:0/2903223959 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8e20083b10 con 0x7f8e340b2ce0 2024-09-13T16:51:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.052+0000 7f8e32a00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e14066760 0x7f8e14068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.052+0000 7f8e32a00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e14066760 0x7f8e14068c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8e24000c00 tx=0x7f8e24008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.056+0000 7f8e2aa00700 1 -- 172.21.15.3:0/2903223959 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8e34158460 con 0x7f8e340b2ce0 2024-09-13T16:51:46.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.236+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e34158840 con 0x7f8e340b2ce0 2024-09-13T16:51:46.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.236+0000 7f8e2aa00700 1 -- 172.21.15.3:0/2903223959 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e34158840 con 0x7f8e340b2ce0 2024-09-13T16:51:46.241 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:51:46.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e14066760 msgr2=0x7f8e14068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e14066760 0x7f8e14068c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8e24000c00 tx=0x7f8e24008030 comp rx=0 tx=0).stop 2024-09-13T16:51:46.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e340b2ce0 msgr2=0x7f8e34154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e340b2ce0 0x7f8e34154450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000c500 tx=0x7f8e2000c8c0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 shutdown_connections 2024-09-13T16:51:46.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8e14066760 0x7f8e14068c20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e341507b0 0x7f8e340b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e340b2ce0 0x7f8e34154450 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 --2- 172.21.15.3:0/2903223959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e3406aaa0 0x7f8e3414cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 >> 172.21.15.3:0/2903223959 conn(0x7f8e340af620 msgr2=0x7f8e340afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:46.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 shutdown_connections 2024-09-13T16:51:46.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.240+0000 7f8e33e00700 1 -- 172.21.15.3:0/2903223959 wait complete. 2024-09-13T16:51:46.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:51:46.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.436+0000 7f74fa400700 1 Processor -- start 2024-09-13T16:51:46.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.436+0000 7f74fa400700 1 -- start start 2024-09-13T16:51:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74fa400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 0x7f74f406d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74fa400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74f40cded0 0x7f74f415b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74fa400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f40b41e0 con 0x7f74f406dcb0 2024-09-13T16:51:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74fa400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f406e0f0 con 0x7f74f40cded0 2024-09-13T16:51:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74fa400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f406a390 con 0x7f74f41442c0 2024-09-13T16:51:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74f9a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74f40cded0 0x7f74f415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74ebe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f74f406dcb0 0x7f74f40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47318/0 (socket says 172.21.15.3:47318) 2024-09-13T16:51:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74ebe00700 1 -- 172.21.15.3:0/289465956 learned_addr learned my addr 172.21.15.3:0/289465956 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74f9000700 1 --2- 172.21.15.3:0/289465956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 0x7f74f406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74f9a00700 1 -- 172.21.15.3:0/289465956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 msgr2=0x7f74f406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74f9a00700 1 --2- 172.21.15.3:0/289465956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 0x7f74f406d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74f9a00700 1 -- 172.21.15.3:0/289465956 >> v1:172.21.15.3:6789/0 conn(0x7f74f406dcb0 legacy=0x7f74f40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74f9a00700 1 -- 172.21.15.3:0/289465956 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f406b1f0 con 0x7f74f40cded0 2024-09-13T16:51:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.440+0000 7f74f9a00700 1 --2- 172.21.15.3:0/289465956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f415b110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009d90 tx=0x7f74ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bad901e3117ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74f3e00700 1 -- 172.21.15.3:0/289465956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00e070 con 0x7f74f40cded0 2024-09-13T16:51:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74f3e00700 1 -- 172.21.15.3:0/289465956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec00bbd0 con 0x7f74f40cded0 2024-09-13T16:51:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74f3e00700 1 -- 172.21.15.3:0/289465956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec012430 con 0x7f74f40cded0 2024-09-13T16:51:46.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74fa400700 1 -- 172.21.15.3:0/289465956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 msgr2=0x7f74f415b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74fa400700 1 --2- 172.21.15.3:0/289465956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f415b110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009d90 tx=0x7f74ec00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:46.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74fa400700 1 -- 172.21.15.3:0/289465956 shutdown_connections 2024-09-13T16:51:46.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74fa400700 1 --2- 172.21.15.3:0/289465956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f415b110 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74fa400700 1 --2- 172.21.15.3:0/289465956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 0x7f74f406d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74fa400700 1 -- 172.21.15.3:0/289465956 >> 172.21.15.3:0/289465956 conn(0x7f74f40af620 msgr2=0x7f74f40afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74fa400700 1 -- 172.21.15.3:0/289465956 shutdown_connections 2024-09-13T16:51:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.444+0000 7f74fa400700 1 -- 172.21.15.3:0/289465956 wait complete. 2024-09-13T16:51:46.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74fa400700 1 Processor -- start 2024-09-13T16:51:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74fa400700 1 -- start start 2024-09-13T16:51:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74fa400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f406dcb0 0x7f74f415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74fa400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f41332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74fa400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 0x7f74f406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74fa400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4130840 con 0x7f74f40cded0 2024-09-13T16:51:46.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74fa400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f413dc10 con 0x7f74f406dcb0 2024-09-13T16:51:46.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74fa400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f415aa50 con 0x7f74f41442c0 2024-09-13T16:51:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74ebe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f41332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74ebe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f41332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38928/0 (socket says 172.21.15.3:38928) 2024-09-13T16:51:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.448+0000 7f74ebe00700 1 -- 172.21.15.3:0/3037060237 learned_addr learned my addr 172.21.15.3:0/3037060237 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74f9000700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f406dcb0 0x7f74f415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74f9a00700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 0x7f74f406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74ebe00700 1 -- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 msgr2=0x7f74f406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74ebe00700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 0x7f74f406fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74ebe00700 1 -- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f406dcb0 msgr2=0x7f74f415a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74ebe00700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f406dcb0 0x7f74f415a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74ebe00700 1 -- 172.21.15.3:0/3037060237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec009a70 con 0x7f74f40cded0 2024-09-13T16:51:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74ebe00700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f41332b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c700 tx=0x7f74e400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74f1600700 1 -- 172.21.15.3:0/3037060237 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4005d90 con 0x7f74f40cded0 2024-09-13T16:51:46.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74f1600700 1 -- 172.21.15.3:0/3037060237 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e40070d0 con 0x7f74f40cded0 2024-09-13T16:51:46.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f41337f0 con 0x7f74f40cded0 2024-09-13T16:51:46.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74f1600700 1 -- 172.21.15.3:0/3037060237 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4007440 con 0x7f74f40cded0 2024-09-13T16:51:46.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f40ad0c0 con 0x7f74f40cded0 2024-09-13T16:51:46.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.452+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f4144730 con 0x7f74f40cded0 2024-09-13T16:51:46.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.456+0000 7f74f1600700 1 -- 172.21.15.3:0/3037060237 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74e40075a0 con 0x7f74f40cded0 2024-09-13T16:51:46.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.456+0000 7f74f1600700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d0066760 0x7f74d0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.456+0000 7f74f1600700 1 -- 172.21.15.3:0/3037060237 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f74e4083870 con 0x7f74f40cded0 2024-09-13T16:51:46.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.456+0000 7f74f9000700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d0066760 0x7f74d0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.460+0000 7f74f9000700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d0066760 0x7f74d0068c20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74dc007a60 tx=0x7f74dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.460+0000 7f74f1600700 1 -- 172.21.15.3:0/3037060237 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74e4050830 con 0x7f74f40cded0 2024-09-13T16:51:46.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.644+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74f415e300 con 0x7f74f40cded0 2024-09-13T16:51:46.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.644+0000 7f74f1600700 1 -- 172.21.15.3:0/3037060237 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74e4053e50 con 0x7f74f40cded0 2024-09-13T16:51:46.651 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:46.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d0066760 msgr2=0x7f74d0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d0066760 0x7f74d0068c20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74dc007a60 tx=0x7f74dc008040 comp rx=0 tx=0).stop 2024-09-13T16:51:46.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 msgr2=0x7f74f41332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f41332b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c700 tx=0x7f74e400ca10 comp rx=0 tx=0).stop 2024-09-13T16:51:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 shutdown_connections 2024-09-13T16:51:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f74d0066760 0x7f74d0068c20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f41442c0 0x7f74f406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f40cded0 0x7f74f41332b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 --2- 172.21.15.3:0/3037060237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f406dcb0 0x7f74f415a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.648+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 >> 172.21.15.3:0/3037060237 conn(0x7f74f40af620 msgr2=0x7f74f40b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.652+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 shutdown_connections 2024-09-13T16:51:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.652+0000 7f74fa400700 1 -- 172.21.15.3:0/3037060237 wait complete. 2024-09-13T16:51:46.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:51:46.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.856+0000 7f0467000700 1 Processor -- start 2024-09-13T16:51:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.856+0000 7f0467000700 1 -- start start 2024-09-13T16:51:46.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f0467000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04601610e0 0x7f046006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f0467000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0460161fe0 con 0x7f0460161bd0 2024-09-13T16:51:46.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f0467000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0460162250 con 0x7f046006feb0 2024-09-13T16:51:46.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f0467000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0460070320 con 0x7f04601610e0 2024-09-13T16:51:46.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04601610e0 0x7f046006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f0465000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0460161bd0 0x7f04601632f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47324/0 (socket says 172.21.15.3:47324) 2024-09-13T16:51:46.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f0465000700 1 -- 172.21.15.3:0/3272533945 learned_addr learned my addr 172.21.15.3:0/3272533945 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:46.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045ea00700 1 -- 172.21.15.3:0/3272533945 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011869958 0 0) 0x7f0460161fe0 con 0x7f0460161bd0 2024-09-13T16:51:46.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045ea00700 1 -- 172.21.15.3:0/3272533945 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04440035d0 con 0x7f0460161bd0 2024-09-13T16:51:46.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045fe00700 1 -- 172.21.15.3:0/3272533945 >> v1:172.21.15.157:6789/0 conn(0x7f046006feb0 legacy=0x7f04601614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045fe00700 1 -- 172.21.15.3:0/3272533945 >> v1:172.21.15.3:6789/0 conn(0x7f0460161bd0 legacy=0x7f04601632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045fe00700 1 -- 172.21.15.3:0/3272533945 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460162530 con 0x7f04601610e0 2024-09-13T16:51:46.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045fe00700 1 --2- 172.21.15.3:0/3272533945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04601610e0 0x7f046006f970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0450009850 tx=0x7f0450009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c9ce8e9909f2fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045ea00700 1 -- 172.21.15.3:0/3272533945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450011830 con 0x7f04601610e0 2024-09-13T16:51:46.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045ea00700 1 -- 172.21.15.3:0/3272533945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450011990 con 0x7f04601610e0 2024-09-13T16:51:46.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.860+0000 7f045ea00700 1 -- 172.21.15.3:0/3272533945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450011d00 con 0x7f04601610e0 2024-09-13T16:51:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 -- 172.21.15.3:0/3272533945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04601610e0 msgr2=0x7f046006f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 --2- 172.21.15.3:0/3272533945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04601610e0 0x7f046006f970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0450009850 tx=0x7f0450009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 -- 172.21.15.3:0/3272533945 shutdown_connections 2024-09-13T16:51:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 --2- 172.21.15.3:0/3272533945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04601610e0 0x7f046006f970 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 -- 172.21.15.3:0/3272533945 >> 172.21.15.3:0/3272533945 conn(0x7f04600b2b00 msgr2=0x7f04600b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 -- 172.21.15.3:0/3272533945 shutdown_connections 2024-09-13T16:51:46.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 -- 172.21.15.3:0/3272533945 wait complete. 2024-09-13T16:51:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 Processor -- start 2024-09-13T16:51:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.864+0000 7f0467000700 1 -- start start 2024-09-13T16:51:46.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0467000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f046006feb0 0x7f0460148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0467000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04601610e0 0x7f0460146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0467000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0460161bd0 0x7f04601473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0467000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04601489c0 con 0x7f0460161bd0 2024-09-13T16:51:46.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0467000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460147db0 con 0x7f046006feb0 2024-09-13T16:51:46.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0467000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0460148090 con 0x7f04601610e0 2024-09-13T16:51:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0460161bd0 0x7f04601473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0460161bd0 0x7f04601473d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38946/0 (socket says 172.21.15.3:38946) 2024-09-13T16:51:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 -- 172.21.15.3:0/1624477770 learned_addr learned my addr 172.21.15.3:0/1624477770 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f045f400700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04601610e0 0x7f0460146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f045fe00700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f046006feb0 0x7f0460148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 -- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04601610e0 msgr2=0x7f0460146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04601610e0 0x7f0460146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 -- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f046006feb0 msgr2=0x7f0460148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f046006feb0 0x7f0460148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 -- 172.21.15.3:0/1624477770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04500094b0 con 0x7f0460161bd0 2024-09-13T16:51:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f045f400700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04601610e0 0x7f0460146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0465000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0460161bd0 0x7f04601473d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f045800caa0 tx=0x7f045800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0457e00700 1 -- 172.21.15.3:0/1624477770 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045800a7b0 con 0x7f0460161bd0 2024-09-13T16:51:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0457e00700 1 -- 172.21.15.3:0/1624477770 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045800a910 con 0x7f0460161bd0 2024-09-13T16:51:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460148370 con 0x7f0460161bd0 2024-09-13T16:51:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0457e00700 1 -- 172.21.15.3:0/1624477770 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045800aca0 con 0x7f0460161bd0 2024-09-13T16:51:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.868+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0460147910 con 0x7f0460161bd0 2024-09-13T16:51:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.872+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0460147b60 con 0x7f0460161bd0 2024-09-13T16:51:46.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.872+0000 7f0457e00700 1 -- 172.21.15.3:0/1624477770 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f04580113a0 con 0x7f0460161bd0 2024-09-13T16:51:46.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.872+0000 7f0457e00700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0440066730 0x7f0440068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:46.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.876+0000 7f0457e00700 1 -- 172.21.15.3:0/1624477770 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0458084080 con 0x7f0460161bd0 2024-09-13T16:51:46.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.876+0000 7f045fe00700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0440066730 0x7f0440068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:46.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.876+0000 7f045fe00700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0440066730 0x7f0440068bf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0450009ba0 tx=0x7f0450008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:46.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:46.884+0000 7f0457e00700 1 -- 172.21.15.3:0/1624477770 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f04580b5db0 con 0x7f0460161bd0 2024-09-13T16:51:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.064+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f046006f970 con 0x7f0460161bd0 2024-09-13T16:51:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.064+0000 7f0457e00700 1 -- 172.21.15.3:0/1624477770 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04580170a0 con 0x7f0460161bd0 2024-09-13T16:51:47.071 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0440066730 msgr2=0x7f0440068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0440066730 0x7f0440068bf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0450009ba0 tx=0x7f0450008030 comp rx=0 tx=0).stop 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0460161bd0 msgr2=0x7f04601473d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0460161bd0 0x7f04601473d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f045800caa0 tx=0x7f045800cb80 comp rx=0 tx=0).stop 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 shutdown_connections 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0440066730 0x7f0440068bf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0460161bd0 0x7f04601473d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04601610e0 0x7f0460146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 --2- 172.21.15.3:0/1624477770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f046006feb0 0x7f0460148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 >> 172.21.15.3:0/1624477770 conn(0x7f04600b2b00 msgr2=0x7f046015fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:47.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 shutdown_connections 2024-09-13T16:51:47.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.068+0000 7f0467000700 1 -- 172.21.15.3:0/1624477770 wait complete. 2024-09-13T16:51:47.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:51:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.264+0000 7f123a600700 1 Processor -- start 2024-09-13T16:51:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.264+0000 7f123a600700 1 -- start start 2024-09-13T16:51:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.264+0000 7f123a600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.264+0000 7f123a600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f123415b740 0x7f123406a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.264+0000 7f123a600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12340afe90 con 0x7f12340b4fc0 2024-09-13T16:51:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.264+0000 7f123a600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12340cfe80 con 0x7f123415b740 2024-09-13T16:51:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.264+0000 7f123a600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12340af880 con 0x7f123415ac50 2024-09-13T16:51:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1233e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f12340b4fc0 0x7f123415b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:37672/0 (socket says 172.21.15.3:37672) 2024-09-13T16:51:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1233e00700 1 -- 172.21.15.3:0/12442110 learned_addr learned my addr 172.21.15.3:0/12442110 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1239200700 1 --2- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1239c00700 1 --2- 172.21.15.3:0/12442110 >> v2:172.21.15.157:3300/0 conn(0x7f123415b740 0x7f123406a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1233400700 1 -- 172.21.15.3:0/12442110 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234628541 0 0) 0x7f12340afe90 con 0x7f12340b4fc0 2024-09-13T16:51:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1233400700 1 -- 172.21.15.3:0/12442110 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f121c0035d0 con 0x7f12340b4fc0 2024-09-13T16:51:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1239c00700 1 -- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 msgr2=0x7f12340cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1239c00700 1 --2- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1239c00700 1 -- 172.21.15.3:0/12442110 >> v1:172.21.15.157:6789/0 conn(0x7f12340b4fc0 legacy=0x7f123415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1239c00700 1 -- 172.21.15.3:0/12442110 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12340cf610 con 0x7f123415b740 2024-09-13T16:51:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1239200700 1 --2- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1239c00700 1 --2- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 0x7f123406a970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f122c009d90 tx=0x7f122c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fea65f6dbef0b724 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1233400700 1 -- 172.21.15.3:0/12442110 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c00e070 con 0x7f123415b740 2024-09-13T16:51:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1233400700 1 -- 172.21.15.3:0/12442110 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122c00bb20 con 0x7f123415b740 2024-09-13T16:51:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.268+0000 7f1233400700 1 -- 172.21.15.3:0/12442110 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c012430 con 0x7f123415b740 2024-09-13T16:51:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 msgr2=0x7f123406a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 --2- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 0x7f123406a970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f122c009d90 tx=0x7f122c00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- 172.21.15.3:0/12442110 shutdown_connections 2024-09-13T16:51:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 --2- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 0x7f123406a970 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 --2- 172.21.15.3:0/12442110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- 172.21.15.3:0/12442110 >> 172.21.15.3:0/12442110 conn(0x7f12340b2d50 msgr2=0x7f12340b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- 172.21.15.3:0/12442110 shutdown_connections 2024-09-13T16:51:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- 172.21.15.3:0/12442110 wait complete. 2024-09-13T16:51:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 Processor -- start 2024-09-13T16:51:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- start start 2024-09-13T16:51:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12340b4fc0 0x7f1234149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 0x7f123414ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234156800 con 0x7f12340b4fc0 2024-09-13T16:51:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12340ce420 con 0x7f123415b740 2024-09-13T16:51:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.272+0000 7f123a600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1234149a80 con 0x7f123415ac50 2024-09-13T16:51:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12340b4fc0 0x7f1234149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12340b4fc0 0x7f1234149430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38956/0 (socket says 172.21.15.3:38956) 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 -- 172.21.15.3:0/51808904 learned_addr learned my addr 172.21.15.3:0/51808904 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239c00700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 0x7f123414ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1233e00700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 -- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 msgr2=0x7f12340cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 -- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 msgr2=0x7f123414ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 0x7f123414ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 -- 172.21.15.3:0/51808904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12280094b0 con 0x7f12340b4fc0 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1233e00700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1239200700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12340b4fc0 0x7f1234149430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f122800ebe0 tx=0x7f1228009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1230c00700 1 -- 172.21.15.3:0/51808904 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122800c910 con 0x7f12340b4fc0 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1230c00700 1 -- 172.21.15.3:0/51808904 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122800ca70 con 0x7f12340b4fc0 2024-09-13T16:51:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f1230c00700 1 -- 172.21.15.3:0/51808904 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122800ce00 con 0x7f12340b4fc0 2024-09-13T16:51:47.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c009a70 con 0x7f12340b4fc0 2024-09-13T16:51:47.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123414a040 con 0x7f12340b4fc0 2024-09-13T16:51:47.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.276+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12340b09f0 con 0x7f12340b4fc0 2024-09-13T16:51:47.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.280+0000 7f1230c00700 1 -- 172.21.15.3:0/51808904 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1228013470 con 0x7f12340b4fc0 2024-09-13T16:51:47.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.280+0000 7f1230c00700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1214066760 0x7f1214068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.280+0000 7f1230c00700 1 -- 172.21.15.3:0/51808904 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1228084850 con 0x7f12340b4fc0 2024-09-13T16:51:47.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.280+0000 7f1233e00700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1214066760 0x7f1214068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.284+0000 7f1233e00700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1214066760 0x7f1214068c20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f12341f5db0 tx=0x7f1220009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:47.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.288+0000 7f1230c00700 1 -- 172.21.15.3:0/51808904 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f12280516d0 con 0x7f12340b4fc0 2024-09-13T16:51:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.468+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f123406b200 con 0x7f12340b4fc0 2024-09-13T16:51:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.468+0000 7f1230c00700 1 -- 172.21.15.3:0/51808904 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1228054cf0 con 0x7f12340b4fc0 2024-09-13T16:51:47.474 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:51:47.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1214066760 msgr2=0x7f1214068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1214066760 0x7f1214068c20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f12341f5db0 tx=0x7f1220009040 comp rx=0 tx=0).stop 2024-09-13T16:51:47.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12340b4fc0 msgr2=0x7f1234149430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12340b4fc0 0x7f1234149430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f122800ebe0 tx=0x7f1228009ed0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 shutdown_connections 2024-09-13T16:51:47.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1214066760 0x7f1214068c20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123415b740 0x7f123414ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123415ac50 0x7f12340cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 --2- 172.21.15.3:0/51808904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12340b4fc0 0x7f1234149430 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 >> 172.21.15.3:0/51808904 conn(0x7f12340b2d50 msgr2=0x7f12340ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:47.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 shutdown_connections 2024-09-13T16:51:47.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.472+0000 7f123a600700 1 -- 172.21.15.3:0/51808904 wait complete. 2024-09-13T16:51:47.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:47.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.668+0000 7f4981000700 1 Processor -- start 2024-09-13T16:51:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.668+0000 7f4981000700 1 -- start start 2024-09-13T16:51:47.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.668+0000 7f4981000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f497c071de0 0x7f497c0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f497c15f0b0 0x7f497c1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497c15bb00 con 0x7f497c0721c0 2024-09-13T16:51:47.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c15bde0 con 0x7f497c15f0b0 2024-09-13T16:51:47.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c15e8d0 con 0x7f497c071de0 2024-09-13T16:51:47.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f497a800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f497c15f0b0 0x7f497c1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4979400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f497c0721c0 0x7f497c15b240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47326/0 (socket says 172.21.15.3:47326) 2024-09-13T16:51:47.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4979400700 1 -- 172.21.15.3:0/3198007639 learned_addr learned my addr 172.21.15.3:0/3198007639 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:47.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4979e00700 1 --2- 172.21.15.3:0/3198007639 >> v2:172.21.15.157:3300/0 conn(0x7f497c071de0 0x7f497c0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f497a800700 1 -- 172.21.15.3:0/3198007639 >> v2:172.21.15.157:3300/0 conn(0x7f497c071de0 msgr2=0x7f497c0cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:47.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f497a800700 1 --2- 172.21.15.3:0/3198007639 >> v2:172.21.15.157:3300/0 conn(0x7f497c071de0 0x7f497c0cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f497a800700 1 -- 172.21.15.3:0/3198007639 >> v1:172.21.15.3:6789/0 conn(0x7f497c0721c0 legacy=0x7f497c15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f497a800700 1 -- 172.21.15.3:0/3198007639 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c15ebb0 con 0x7f497c15f0b0 2024-09-13T16:51:47.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4978a00700 1 -- 172.21.15.3:0/3198007639 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114308476 0 0) 0x7f497c15bb00 con 0x7f497c0721c0 2024-09-13T16:51:47.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f497a800700 1 --2- 172.21.15.3:0/3198007639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c15f0b0 0x7f497c1581f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f497c06b400 tx=0x7f497400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b4cc9cb6f9df70 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:47.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4978a00700 1 -- 172.21.15.3:0/3198007639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400e070 con 0x7f497c15f0b0 2024-09-13T16:51:47.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4978a00700 1 -- 172.21.15.3:0/3198007639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400bb20 con 0x7f497c15f0b0 2024-09-13T16:51:47.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4978a00700 1 -- 172.21.15.3:0/3198007639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974012430 con 0x7f497c15f0b0 2024-09-13T16:51:47.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 -- 172.21.15.3:0/3198007639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c15f0b0 msgr2=0x7f497c1581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 --2- 172.21.15.3:0/3198007639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c15f0b0 0x7f497c1581f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f497c06b400 tx=0x7f497400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:47.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 -- 172.21.15.3:0/3198007639 shutdown_connections 2024-09-13T16:51:47.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 --2- 172.21.15.3:0/3198007639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c15f0b0 0x7f497c1581f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 --2- 172.21.15.3:0/3198007639 >> v2:172.21.15.157:3300/0 conn(0x7f497c071de0 0x7f497c0cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.672+0000 7f4981000700 1 -- 172.21.15.3:0/3198007639 >> 172.21.15.3:0/3198007639 conn(0x7f497c071050 msgr2=0x7f497c071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:47.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.676+0000 7f4981000700 1 -- 172.21.15.3:0/3198007639 shutdown_connections 2024-09-13T16:51:47.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.676+0000 7f4981000700 1 -- 172.21.15.3:0/3198007639 wait complete. 2024-09-13T16:51:47.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.676+0000 7f4981000700 1 Processor -- start 2024-09-13T16:51:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.676+0000 7f4981000700 1 -- start start 2024-09-13T16:51:47.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4981000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c071de0 0x7f497c1f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4981000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0721c0 0x7f497c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4981000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c15f0b0 0x7f497c1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4981000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c1f6ce0 con 0x7f497c071de0 2024-09-13T16:51:47.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4981000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c1f6310 con 0x7f497c0721c0 2024-09-13T16:51:47.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4981000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c0c91e0 con 0x7f497c15f0b0 2024-09-13T16:51:47.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c071de0 0x7f497c1f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c071de0 0x7f497c1f67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38984/0 (socket says 172.21.15.3:38984) 2024-09-13T16:51:47.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 -- 172.21.15.3:0/2490083009 learned_addr learned my addr 172.21.15.3:0/2490083009 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979400700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0721c0 0x7f497c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f497a800700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c15f0b0 0x7f497c1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 -- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c15f0b0 msgr2=0x7f497c1f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c15f0b0 0x7f497c1f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 -- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0721c0 msgr2=0x7f497c0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0721c0 0x7f497c0c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 -- 172.21.15.3:0/2490083009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974009a70 con 0x7f497c071de0 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4979e00700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c071de0 0x7f497c1f67a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f496c00e9d0 tx=0x7f496c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4972000700 1 -- 172.21.15.3:0/2490083009 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c009720 con 0x7f497c071de0 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4972000700 1 -- 172.21.15.3:0/2490083009 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c00c850 con 0x7f497c071de0 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4972000700 1 -- 172.21.15.3:0/2490083009 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00cbc0 con 0x7f497c071de0 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.680+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c1f5d00 con 0x7f497c071de0 2024-09-13T16:51:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.684+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c0c9760 con 0x7f497c071de0 2024-09-13T16:51:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.684+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c158710 con 0x7f497c071de0 2024-09-13T16:51:47.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.684+0000 7f4972000700 1 -- 172.21.15.3:0/2490083009 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f496c00cd20 con 0x7f497c071de0 2024-09-13T16:51:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.688+0000 7f4972000700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f495c066730 0x7f495c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.688+0000 7f4972000700 1 -- 172.21.15.3:0/2490083009 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f496c084610 con 0x7f497c071de0 2024-09-13T16:51:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.688+0000 7f4979400700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f495c066730 0x7f495c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.692+0000 7f4979400700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f495c066730 0x7f495c068bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f497c0cca20 tx=0x7f4968007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.696+0000 7f4972000700 1 -- 172.21.15.3:0/2490083009 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f497c158710 con 0x7f497c071de0 2024-09-13T16:51:47.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.924+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f497c158710 con 0x7f497c071de0 2024-09-13T16:51:47.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.924+0000 7f4972000700 1 -- 172.21.15.3:0/2490083009 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f497c158710 con 0x7f497c071de0 2024-09-13T16:51:47.931 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:47.931 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:55.700655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:51:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f495c066730 msgr2=0x7f495c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f495c066730 0x7f495c068bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f497c0cca20 tx=0x7f4968007c00 comp rx=0 tx=0).stop 2024-09-13T16:51:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c071de0 msgr2=0x7f497c1f67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c071de0 0x7f497c1f67a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f496c00e9d0 tx=0x7f496c00ed90 comp rx=0 tx=0).stop 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 shutdown_connections 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f495c066730 0x7f495c068bf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c15f0b0 0x7f497c1f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0721c0 0x7f497c0c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 --2- 172.21.15.3:0/2490083009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f497c071de0 0x7f497c1f67a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 >> 172.21.15.3:0/2490083009 conn(0x7f497c071050 msgr2=0x7f497c06e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 shutdown_connections 2024-09-13T16:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:47.928+0000 7f4981000700 1 -- 172.21.15.3:0/2490083009 wait complete. 2024-09-13T16:51:47.936 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-13T16:51:47.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:48.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.128+0000 7f37d7200700 1 Processor -- start 2024-09-13T16:51:48.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.128+0000 7f37d7200700 1 -- start start 2024-09-13T16:51:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.128+0000 7f37d7200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d81443b0 0x7f37d80b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.128+0000 7f37d7200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f37d806ccf0 0x7f37d815b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.128+0000 7f37d7200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d80ce8e0 con 0x7f37d8144790 2024-09-13T16:51:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.128+0000 7f37d7200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d80ccd30 con 0x7f37d806ccf0 2024-09-13T16:51:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.128+0000 7f37d7200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80cd010 con 0x7f37d81443b0 2024-09-13T16:51:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37cd400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f37d8144790 0x7f37d806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47334/0 (socket says 172.21.15.3:47334) 2024-09-13T16:51:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37cd400700 1 -- 172.21.15.3:0/730645989 learned_addr learned my addr 172.21.15.3:0/730645989 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d6800700 1 --2- 172.21.15.3:0/730645989 >> v2:172.21.15.157:3300/0 conn(0x7f37d806ccf0 0x7f37d815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d5e00700 1 --2- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d81443b0 0x7f37d80b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d6800700 1 -- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d81443b0 msgr2=0x7f37d80b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:48.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d6800700 1 --2- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d81443b0 0x7f37d80b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d5400700 1 -- 172.21.15.3:0/730645989 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146228294 0 0) 0x7f37d80ce8e0 con 0x7f37d8144790 2024-09-13T16:51:48.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d6800700 1 -- 172.21.15.3:0/730645989 >> v1:172.21.15.3:6789/0 conn(0x7f37d8144790 legacy=0x7f37d806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d6800700 1 -- 172.21.15.3:0/730645989 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80cd2f0 con 0x7f37d806ccf0 2024-09-13T16:51:48.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d5e00700 1 --2- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d81443b0 0x7f37d80b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:48.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d6800700 1 --2- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 0x7f37d815b220 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009d90 tx=0x7f37d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c3bc501ff21785 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:48.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d5400700 1 -- 172.21.15.3:0/730645989 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d000e070 con 0x7f37d806ccf0 2024-09-13T16:51:48.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d5400700 1 -- 172.21.15.3:0/730645989 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d000bb20 con 0x7f37d806ccf0 2024-09-13T16:51:48.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.132+0000 7f37d5400700 1 -- 172.21.15.3:0/730645989 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0012430 con 0x7f37d806ccf0 2024-09-13T16:51:48.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 -- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 msgr2=0x7f37d815b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 --2- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 0x7f37d815b220 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009d90 tx=0x7f37d000b330 comp rx=0 tx=0).stop 2024-09-13T16:51:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 -- 172.21.15.3:0/730645989 shutdown_connections 2024-09-13T16:51:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 --2- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 0x7f37d815b220 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 --2- 172.21.15.3:0/730645989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d81443b0 0x7f37d80b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 -- 172.21.15.3:0/730645989 >> 172.21.15.3:0/730645989 conn(0x7f37d80b0cf0 msgr2=0x7f37d80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:48.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 -- 172.21.15.3:0/730645989 shutdown_connections 2024-09-13T16:51:48.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 -- 172.21.15.3:0/730645989 wait complete. 2024-09-13T16:51:48.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 Processor -- start 2024-09-13T16:51:48.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.136+0000 7f37d7200700 1 -- start start 2024-09-13T16:51:48.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 0x7f37d81f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d7200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37d81443b0 0x7f37d813e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d7200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d813cfb0 0x7f37d813b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d7200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81f6ed0 con 0x7f37d81443b0 2024-09-13T16:51:48.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d813f1d0 con 0x7f37d806ccf0 2024-09-13T16:51:48.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d7200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d813ef10 con 0x7f37d813cfb0 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37d81443b0 0x7f37d813e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37d81443b0 0x7f37d813e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39000/0 (socket says 172.21.15.3:39000) 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 -- 172.21.15.3:0/2519531722 learned_addr learned my addr 172.21.15.3:0/2519531722 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d5e00700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 0x7f37d81f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d6800700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d813cfb0 0x7f37d813b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 -- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d813cfb0 msgr2=0x7f37d813b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d813cfb0 0x7f37d813b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 -- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 msgr2=0x7f37d81f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 0x7f37d81f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 -- 172.21.15.3:0/2519531722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80094b0 con 0x7f37d81443b0 2024-09-13T16:51:48.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cd400700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37d81443b0 0x7f37d813e9d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c4e0 tx=0x7f37c000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:48.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cea00700 1 -- 172.21.15.3:0/2519531722 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c0004ca0 con 0x7f37d81443b0 2024-09-13T16:51:48.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cea00700 1 -- 172.21.15.3:0/2519531722 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c0004e20 con 0x7f37d81443b0 2024-09-13T16:51:48.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d0009a70 con 0x7f37d81443b0 2024-09-13T16:51:48.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.140+0000 7f37cea00700 1 -- 172.21.15.3:0/2519531722 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c0007310 con 0x7f37d81443b0 2024-09-13T16:51:48.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.144+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d813c4d0 con 0x7f37d81443b0 2024-09-13T16:51:48.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.144+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d80b39f0 con 0x7f37d81443b0 2024-09-13T16:51:48.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.148+0000 7f37cea00700 1 -- 172.21.15.3:0/2519531722 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37c0007470 con 0x7f37d81443b0 2024-09-13T16:51:48.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.148+0000 7f37cea00700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f37b4066730 0x7f37b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.148+0000 7f37cea00700 1 -- 172.21.15.3:0/2519531722 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f37c0084110 con 0x7f37d81443b0 2024-09-13T16:51:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.148+0000 7f37d5e00700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f37b4066730 0x7f37b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.148+0000 7f37d5e00700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f37b4066730 0x7f37b4068bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009d80 tx=0x7f37c8009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.156+0000 7f37cea00700 1 -- 172.21.15.3:0/2519531722 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f37c00510d0 con 0x7f37d81443b0 2024-09-13T16:51:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.348+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37d80b2740 con 0x7f37d81443b0 2024-09-13T16:51:48.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37cea00700 1 -- 172.21.15.3:0/2519531722 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f37c0007720 con 0x7f37d81443b0 2024-09-13T16:51:48.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:48.357 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:48.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f37b4066730 msgr2=0x7f37b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f37b4066730 0x7f37b4068bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009d80 tx=0x7f37c8009240 comp rx=0 tx=0).stop 2024-09-13T16:51:48.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37d81443b0 msgr2=0x7f37d813e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37d81443b0 0x7f37d813e9d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c4e0 tx=0x7f37c000c7f0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 shutdown_connections 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f37b4066730 0x7f37b4068bf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37d813cfb0 0x7f37d813b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37d81443b0 0x7f37d813e9d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 --2- 172.21.15.3:0/2519531722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37d806ccf0 0x7f37d81f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.352+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 >> 172.21.15.3:0/2519531722 conn(0x7f37d80b0cf0 msgr2=0x7f37d8141360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.356+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 shutdown_connections 2024-09-13T16:51:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.356+0000 7f37d7200700 1 -- 172.21.15.3:0/2519531722 wait complete. 2024-09-13T16:51:48.384 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T16:51:48.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:48.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.560+0000 7fe026400700 1 Processor -- start 2024-09-13T16:51:48.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.560+0000 7fe026400700 1 -- start start 2024-09-13T16:51:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe026400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0200ce520 0x7fe020158180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe026400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0200cc800 0x7fe0201586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe026400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe020158c00 con 0x7fe0200cc420 2024-09-13T16:51:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe026400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02006d4c0 con 0x7fe0200cc800 2024-09-13T16:51:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe026400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe020071320 con 0x7fe0200ce520 2024-09-13T16:51:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0200cc800 0x7fe0201586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0200cc800 0x7fe0201586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39004/0 (socket says 172.21.15.3:39004) 2024-09-13T16:51:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01fe00700 1 -- 172.21.15.3:0/4175692794 learned_addr learned my addr 172.21.15.3:0/4175692794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01ea00700 1 --2- 172.21.15.3:0/4175692794 >> v2:172.21.15.157:3300/0 conn(0x7fe0200ce520 0x7fe020158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01e000700 1 -- 172.21.15.3:0/4175692794 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407358224 0 0) 0x7fe020158c00 con 0x7fe0200cc420 2024-09-13T16:51:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01e000700 1 -- 172.21.15.3:0/4175692794 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0040035d0 con 0x7fe0200cc420 2024-09-13T16:51:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01fe00700 1 -- 172.21.15.3:0/4175692794 >> v2:172.21.15.157:3300/0 conn(0x7fe0200ce520 msgr2=0x7fe020158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01fe00700 1 --2- 172.21.15.3:0/4175692794 >> v2:172.21.15.157:3300/0 conn(0x7fe0200ce520 0x7fe020158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.564+0000 7fe01fe00700 1 -- 172.21.15.3:0/4175692794 >> v1:172.21.15.157:6789/0 conn(0x7fe0200cc420 legacy=0x7fe02015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe01fe00700 1 -- 172.21.15.3:0/4175692794 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020071600 con 0x7fe0200cc800 2024-09-13T16:51:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe01ea00700 1 --2- 172.21.15.3:0/4175692794 >> v2:172.21.15.157:3300/0 conn(0x7fe0200ce520 0x7fe020158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:48.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe01fe00700 1 --2- 172.21.15.3:0/4175692794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe0201586c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe02015b1a0 tx=0x7fe01800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986018f7b8908788 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:48.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe01e000700 1 -- 172.21.15.3:0/4175692794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01800e070 con 0x7fe0200cc800 2024-09-13T16:51:48.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe01e000700 1 -- 172.21.15.3:0/4175692794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01800b910 con 0x7fe0200cc800 2024-09-13T16:51:48.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe01e000700 1 -- 172.21.15.3:0/4175692794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01800bcc0 con 0x7fe0200cc800 2024-09-13T16:51:48.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe026400700 1 -- 172.21.15.3:0/4175692794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 msgr2=0x7fe0201586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe026400700 1 --2- 172.21.15.3:0/4175692794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe0201586c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe02015b1a0 tx=0x7fe01800b330 comp rx=0 tx=0).stop 2024-09-13T16:51:48.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe026400700 1 -- 172.21.15.3:0/4175692794 shutdown_connections 2024-09-13T16:51:48.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe026400700 1 --2- 172.21.15.3:0/4175692794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe0201586c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe026400700 1 --2- 172.21.15.3:0/4175692794 >> v2:172.21.15.157:3300/0 conn(0x7fe0200ce520 0x7fe020158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe026400700 1 -- 172.21.15.3:0/4175692794 >> 172.21.15.3:0/4175692794 conn(0x7fe02006a390 msgr2=0x7fe02006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe026400700 1 -- 172.21.15.3:0/4175692794 shutdown_connections 2024-09-13T16:51:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.568+0000 7fe026400700 1 -- 172.21.15.3:0/4175692794 wait complete. 2024-09-13T16:51:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe026400700 1 Processor -- start 2024-09-13T16:51:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe026400700 1 -- start start 2024-09-13T16:51:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe026400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200cc420 0x7fe020152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe026400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe020150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe026400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200ce520 0x7fe0201efb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe026400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200c9420 con 0x7fe0200cc800 2024-09-13T16:51:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe026400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe020150980 con 0x7fe0200ce520 2024-09-13T16:51:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe026400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0201f2cf0 con 0x7fe0200cc420 2024-09-13T16:51:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe01ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe020150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe01ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe020150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39006/0 (socket says 172.21.15.3:39006) 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.572+0000 7fe01ea00700 1 -- 172.21.15.3:0/1998415314 learned_addr learned my addr 172.21.15.3:0/1998415314 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01fe00700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200ce520 0x7fe0201efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01f400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200cc420 0x7fe020152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01ea00700 1 -- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200cc420 msgr2=0x7fe020152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01ea00700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200cc420 0x7fe020152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01ea00700 1 -- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200ce520 msgr2=0x7fe0201efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01ea00700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200ce520 0x7fe0201efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01ea00700 1 -- 172.21.15.3:0/1998415314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c0094b0 con 0x7fe0200cc800 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01f400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200cc420 0x7fe020152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe01ea00700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe020150440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00ec10 tx=0x7fe00c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe017400700 1 -- 172.21.15.3:0/1998415314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00c850 con 0x7fe0200cc800 2024-09-13T16:51:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe017400700 1 -- 172.21.15.3:0/1998415314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c00c9b0 con 0x7fe0200cc800 2024-09-13T16:51:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe017400700 1 -- 172.21.15.3:0/1998415314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00cd40 con 0x7fe0200cc800 2024-09-13T16:51:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe018009b30 con 0x7fe0200cc800 2024-09-13T16:51:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0201f32b0 con 0x7fe0200cc800 2024-09-13T16:51:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.576+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0200c9590 con 0x7fe0200cc800 2024-09-13T16:51:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.580+0000 7fe017400700 1 -- 172.21.15.3:0/1998415314 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe00c013470 con 0x7fe0200cc800 2024-09-13T16:51:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.580+0000 7fe017400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe000066730 0x7fe000068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.580+0000 7fe017400700 1 -- 172.21.15.3:0/1998415314 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe00c0844d0 con 0x7fe0200cc800 2024-09-13T16:51:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.580+0000 7fe01f400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe000066730 0x7fe000068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.580+0000 7fe01f400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe000066730 0x7fe000068bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe0100011e0 tx=0x7fe010009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.588+0000 7fe017400700 1 -- 172.21.15.3:0/1998415314 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe0200c9590 con 0x7fe0200cc800 2024-09-13T16:51:48.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.816+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0200c9560 con 0x7fe0200cc800 2024-09-13T16:51:48.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.816+0000 7fe017400700 1 -- 172.21.15.3:0/1998415314 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4727 (secure 0 0 0) 0x7fe00c051510 con 0x7fe0200cc800 2024-09-13T16:51:48.823 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:48.823 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:55.700655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:51:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe000066730 msgr2=0x7fe000068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe000066730 0x7fe000068bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe0100011e0 tx=0x7fe010009040 comp rx=0 tx=0).stop 2024-09-13T16:51:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 msgr2=0x7fe020150440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe020150440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00ec10 tx=0x7fe00c00ef20 comp rx=0 tx=0).stop 2024-09-13T16:51:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 shutdown_connections 2024-09-13T16:51:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe000066730 0x7fe000068bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200ce520 0x7fe0201efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0200cc800 0x7fe020150440 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 --2- 172.21.15.3:0/1998415314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200cc420 0x7fe020152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 >> 172.21.15.3:0/1998415314 conn(0x7fe02006a390 msgr2=0x7fe02015e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 shutdown_connections 2024-09-13T16:51:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:48.820+0000 7fe026400700 1 -- 172.21.15.3:0/1998415314 wait complete. 2024-09-13T16:51:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-13T16:51:48.848 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.024+0000 7f21b3200700 1 Processor -- start 2024-09-13T16:51:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.024+0000 7f21b3200700 1 -- start start 2024-09-13T16:51:49.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b3200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f21ac06fb50 0x7f21ac0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b3200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21ac0b4560 0x7f21ac15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b3200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac0cf070 con 0x7f21ac06ff30 2024-09-13T16:51:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b3200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac0b4be0 con 0x7f21ac0b4560 2024-09-13T16:51:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b3200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac0cf330 con 0x7f21ac06fb50 2024-09-13T16:51:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b2800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21ac0b4560 0x7f21ac15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b2800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21ac0b4560 0x7f21ac15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39020/0 (socket says 172.21.15.3:39020) 2024-09-13T16:51:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b2800700 1 -- 172.21.15.3:0/2232561114 learned_addr learned my addr 172.21.15.3:0/2232561114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b1e00700 1 --2- 172.21.15.3:0/2232561114 >> v2:172.21.15.157:3300/0 conn(0x7f21ac06fb50 0x7f21ac0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b0a00700 1 -- 172.21.15.3:0/2232561114 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469156893 0 0) 0x7f21ac0cf070 con 0x7f21ac06ff30 2024-09-13T16:51:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b0a00700 1 -- 172.21.15.3:0/2232561114 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21940035d0 con 0x7f21ac06ff30 2024-09-13T16:51:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b2800700 1 -- 172.21.15.3:0/2232561114 >> v2:172.21.15.157:3300/0 conn(0x7f21ac06fb50 msgr2=0x7f21ac0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b2800700 1 --2- 172.21.15.3:0/2232561114 >> v2:172.21.15.157:3300/0 conn(0x7f21ac06fb50 0x7f21ac0cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b2800700 1 -- 172.21.15.3:0/2232561114 >> v1:172.21.15.157:6789/0 conn(0x7f21ac06ff30 legacy=0x7f21ac0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.028+0000 7f21b2800700 1 -- 172.21.15.3:0/2232561114 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0cf610 con 0x7f21ac0b4560 2024-09-13T16:51:49.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b2800700 1 --2- 172.21.15.3:0/2232561114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac0b4560 0x7f21ac15a500 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0d1ba0 tx=0x7f21a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38eccc0820dc11a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b0a00700 1 -- 172.21.15.3:0/2232561114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800e070 con 0x7f21ac0b4560 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b0a00700 1 -- 172.21.15.3:0/2232561114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a800b910 con 0x7f21ac0b4560 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b0a00700 1 -- 172.21.15.3:0/2232561114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800bcc0 con 0x7f21ac0b4560 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b3200700 1 -- 172.21.15.3:0/2232561114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac0b4560 msgr2=0x7f21ac15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b3200700 1 --2- 172.21.15.3:0/2232561114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac0b4560 0x7f21ac15a500 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0d1ba0 tx=0x7f21a800b330 comp rx=0 tx=0).stop 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b3200700 1 -- 172.21.15.3:0/2232561114 shutdown_connections 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b3200700 1 --2- 172.21.15.3:0/2232561114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac0b4560 0x7f21ac15a500 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b3200700 1 --2- 172.21.15.3:0/2232561114 >> v2:172.21.15.157:3300/0 conn(0x7f21ac06fb50 0x7f21ac0cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b3200700 1 -- 172.21.15.3:0/2232561114 >> 172.21.15.3:0/2232561114 conn(0x7f21ac0b2910 msgr2=0x7f21ac0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b3200700 1 -- 172.21.15.3:0/2232561114 shutdown_connections 2024-09-13T16:51:49.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.032+0000 7f21b3200700 1 -- 172.21.15.3:0/2232561114 wait complete. 2024-09-13T16:51:49.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b3200700 1 Processor -- start 2024-09-13T16:51:49.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b3200700 1 -- start start 2024-09-13T16:51:49.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b3200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac06fb50 0x7f21ac06be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b3200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21ac06ff30 0x7f21ac1567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b3200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21ac0b4560 0x7f21ac156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b3200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac06d150 con 0x7f21ac06fb50 2024-09-13T16:51:49.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b3200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac06d430 con 0x7f21ac0b4560 2024-09-13T16:51:49.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b3200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac06d710 con 0x7f21ac06ff30 2024-09-13T16:51:49.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac06fb50 0x7f21ac06be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac06fb50 0x7f21ac06be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39028/0 (socket says 172.21.15.3:39028) 2024-09-13T16:51:49.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1e00700 1 -- 172.21.15.3:0/2415684989 learned_addr learned my addr 172.21.15.3:0/2415684989 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:49.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1400700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21ac06ff30 0x7f21ac1567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b2800700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21ac0b4560 0x7f21ac156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1e00700 1 -- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21ac06ff30 msgr2=0x7f21ac1567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1e00700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21ac06ff30 0x7f21ac1567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1e00700 1 -- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21ac0b4560 msgr2=0x7f21ac156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1e00700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21ac0b4560 0x7f21ac156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.036+0000 7f21b1e00700 1 -- 172.21.15.3:0/2415684989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8009b30 con 0x7f21ac06fb50 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.040+0000 7f21b1e00700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac06fb50 0x7f21ac06be70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f21a000e9d0 tx=0x7f21a000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.040+0000 7f21a6000700 1 -- 172.21.15.3:0/2415684989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0009720 con 0x7f21ac06fb50 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.040+0000 7f21a6000700 1 -- 172.21.15.3:0/2415684989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a000ee70 con 0x7f21ac06fb50 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.040+0000 7f21a6000700 1 -- 172.21.15.3:0/2415684989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000ca50 con 0x7f21ac06fb50 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.040+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac06d9f0 con 0x7f21ac06fb50 2024-09-13T16:51:49.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.040+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac1f2790 con 0x7f21ac06fb50 2024-09-13T16:51:49.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.040+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac0ce960 con 0x7f21ac06fb50 2024-09-13T16:51:49.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.044+0000 7f21a6000700 1 -- 172.21.15.3:0/2415684989 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21a000cbb0 con 0x7f21ac06fb50 2024-09-13T16:51:49.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.044+0000 7f21a6000700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2190066730 0x7f2190068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.044+0000 7f21a6000700 1 -- 172.21.15.3:0/2415684989 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f21a0084500 con 0x7f21ac06fb50 2024-09-13T16:51:49.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.044+0000 7f21b1400700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2190066730 0x7f2190068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.044+0000 7f21b1400700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2190066730 0x7f2190068bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f21ac06cb20 tx=0x7f219c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.052+0000 7f21a6000700 1 -- 172.21.15.3:0/2415684989 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f21a00514a0 con 0x7f21ac06fb50 2024-09-13T16:51:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.284+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21ac0cfe10 con 0x7f21ac06fb50 2024-09-13T16:51:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.284+0000 7f21a6000700 1 -- 172.21.15.3:0/2415684989 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f21a000ce60 con 0x7f21ac06fb50 2024-09-13T16:51:49.291 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:49.291 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/378069778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":378069778},{"type":"v1","addr":"172.21.15.3:6835","nonce":378069778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1206700950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1206700950},{"type":"v1","addr":"172.21.15.157:6836","nonce":1206700950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4281214392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4281214392},{"type":"v1","addr":"172.21.15.3:6833","nonce":4281214392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:50:30.569564+0000","modified":"2024-09-13T16:50:55.700655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/3884968743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3884968743},{"type":"v1","addr":"172.21.15.157:6837","nonce":3884968743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2190066730 msgr2=0x7f2190068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2190066730 0x7f2190068bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f21ac06cb20 tx=0x7f219c009040 comp rx=0 tx=0).stop 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac06fb50 msgr2=0x7f21ac06be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac06fb50 0x7f21ac06be70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f21a000e9d0 tx=0x7f21a000ece0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 shutdown_connections 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2190066730 0x7f2190068bf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21ac0b4560 0x7f21ac156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21ac06ff30 0x7f21ac1567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 --2- 172.21.15.3:0/2415684989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ac06fb50 0x7f21ac06be70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 >> 172.21.15.3:0/2415684989 conn(0x7f21ac0b2910 msgr2=0x7f21ac0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 shutdown_connections 2024-09-13T16:51:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.288+0000 7f21b3200700 1 -- 172.21.15.3:0/2415684989 wait complete. 2024-09-13T16:51:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-13T16:51:49.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.480+0000 7f0df9a00700 1 Processor -- start 2024-09-13T16:51:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.480+0000 7f0df9a00700 1 -- start start 2024-09-13T16:51:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df9a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df40ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df9a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df406a390 con 0x7f0df40ae1e0 2024-09-13T16:51:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df9a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df406a670 con 0x7f0df415cbc0 2024-09-13T16:51:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df9a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df406a950 con 0x7f0df415cfa0 2024-09-13T16:51:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df9000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0df40ae1e0 0x7f0df40cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47342/0 (socket says 172.21.15.3:47342) 2024-09-13T16:51:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df9000700 1 -- 172.21.15.3:0/119375922 learned_addr learned my addr 172.21.15.3:0/119375922 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df3400700 1 --2- 172.21.15.3:0/119375922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df40ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df2a00700 1 -- 172.21.15.3:0/119375922 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515610999 0 0) 0x7f0df406a390 con 0x7f0df40ae1e0 2024-09-13T16:51:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df2a00700 1 -- 172.21.15.3:0/119375922 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd80035d0 con 0x7f0df40ae1e0 2024-09-13T16:51:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df3400700 1 -- 172.21.15.3:0/119375922 >> v1:172.21.15.157:6789/0 conn(0x7f0df415cbc0 legacy=0x7f0df40adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df3400700 1 -- 172.21.15.3:0/119375922 >> v1:172.21.15.3:6789/0 conn(0x7f0df40ae1e0 legacy=0x7f0df40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df3400700 1 -- 172.21.15.3:0/119375922 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df406ac30 con 0x7f0df415cfa0 2024-09-13T16:51:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.484+0000 7f0df3400700 1 --2- 172.21.15.3:0/119375922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df40ccb60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009880 tx=0x7f0de0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e643af15337a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df2a00700 1 -- 172.21.15.3:0/119375922 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0011830 con 0x7f0df415cfa0 2024-09-13T16:51:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df2a00700 1 -- 172.21.15.3:0/119375922 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0011990 con 0x7f0df415cfa0 2024-09-13T16:51:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df2a00700 1 -- 172.21.15.3:0/119375922 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0011d20 con 0x7f0df415cfa0 2024-09-13T16:51:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 -- 172.21.15.3:0/119375922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 msgr2=0x7f0df40ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 --2- 172.21.15.3:0/119375922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df40ccb60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009880 tx=0x7f0de0009b90 comp rx=0 tx=0).stop 2024-09-13T16:51:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 -- 172.21.15.3:0/119375922 shutdown_connections 2024-09-13T16:51:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 --2- 172.21.15.3:0/119375922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df40ccb60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 -- 172.21.15.3:0/119375922 >> 172.21.15.3:0/119375922 conn(0x7f0df40b0cf0 msgr2=0x7f0df40b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 -- 172.21.15.3:0/119375922 shutdown_connections 2024-09-13T16:51:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 -- 172.21.15.3:0/119375922 wait complete. 2024-09-13T16:51:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 Processor -- start 2024-09-13T16:51:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.488+0000 7f0df9a00700 1 -- start start 2024-09-13T16:51:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df40ae1e0 0x7f0df414d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df415cbc0 0x7f0df414f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df415bbc0 con 0x7f0df415cbc0 2024-09-13T16:51:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df415dd90 con 0x7f0df40ae1e0 2024-09-13T16:51:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df415b620 con 0x7f0df415cfa0 2024-09-13T16:51:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df415cbc0 0x7f0df414f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df415cbc0 0x7f0df414f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39044/0 (socket says 172.21.15.3:39044) 2024-09-13T16:51:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 -- 172.21.15.3:0/3063739001 learned_addr learned my addr 172.21.15.3:0/3063739001 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3e00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df40ae1e0 0x7f0df414d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9000700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 -- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 msgr2=0x7f0df415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df415a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 -- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df40ae1e0 msgr2=0x7f0df414d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df40ae1e0 0x7f0df414d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 -- 172.21.15.3:0/3063739001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de00094b0 con 0x7f0df415cbc0 2024-09-13T16:51:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3e00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df40ae1e0 0x7f0df414d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df3400700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df415cbc0 0x7f0df414f690 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0de0013040 tx=0x7f0de0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0debe00700 1 -- 172.21.15.3:0/3063739001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de00036c0 con 0x7f0df415cbc0 2024-09-13T16:51:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0debe00700 1 -- 172.21.15.3:0/3063739001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0003820 con 0x7f0df415cbc0 2024-09-13T16:51:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df415fca0 con 0x7f0df415cbc0 2024-09-13T16:51:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0debe00700 1 -- 172.21.15.3:0/3063739001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0003b90 con 0x7f0df415cbc0 2024-09-13T16:51:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.492+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df415b8e0 con 0x7f0df415cbc0 2024-09-13T16:51:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.496+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df415df00 con 0x7f0df415cbc0 2024-09-13T16:51:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.496+0000 7f0debe00700 1 -- 172.21.15.3:0/3063739001 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0de0003cf0 con 0x7f0df415cbc0 2024-09-13T16:51:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.500+0000 7f0debe00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0dd4066730 0x7f0dd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.500+0000 7f0debe00700 1 -- 172.21.15.3:0/3063739001 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0de00854e0 con 0x7f0df415cbc0 2024-09-13T16:51:49.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.500+0000 7f0df3e00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0dd4066730 0x7f0dd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.500+0000 7f0df3e00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0dd4066730 0x7f0dd4068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0de4000f60 tx=0x7f0de4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.504+0000 7f0debe00700 1 -- 172.21.15.3:0/3063739001 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0df415df00 con 0x7f0df415cbc0 2024-09-13T16:51:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.692+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0df415df00 con 0x7f0df415cbc0 2024-09-13T16:51:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.692+0000 7f0debe00700 1 -- 172.21.15.3:0/3063739001 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18631 (secure 0 0 0) 0x7f0df415df00 con 0x7f0df415cbc0 2024-09-13T16:51:49.700 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:49.700 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:15.057537+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:50:28.793699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:50:29.563248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:50:32.649835+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0dd4066730 msgr2=0x7f0dd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0dd4066730 0x7f0dd4068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0de4000f60 tx=0x7f0de4009040 comp rx=0 tx=0).stop 2024-09-13T16:51:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df415cbc0 msgr2=0x7f0df414f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df415cbc0 0x7f0df414f690 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0de0013040 tx=0x7f0de0004770 comp rx=0 tx=0).stop 2024-09-13T16:51:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 shutdown_connections 2024-09-13T16:51:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0dd4066730 0x7f0dd4068bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df415cfa0 0x7f0df415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df415cbc0 0x7f0df414f690 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 --2- 172.21.15.3:0/3063739001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df40ae1e0 0x7f0df414d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 >> 172.21.15.3:0/3063739001 conn(0x7f0df40b0cf0 msgr2=0x7f0df40ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 shutdown_connections 2024-09-13T16:51:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.696+0000 7f0df9a00700 1 -- 172.21.15.3:0/3063739001 wait complete. 2024-09-13T16:51:49.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T16:51:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.904+0000 7f8837200700 1 Processor -- start 2024-09-13T16:51:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.904+0000 7f8837200700 1 -- start start 2024-09-13T16:51:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.908+0000 7f8837200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806fb50 0x7f88380cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.908+0000 7f8837200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f883806ff30 0x7f88380b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.908+0000 7f8837200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88380b4390 0x7f883815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.908+0000 7f8837200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88380cf070 con 0x7f883806ff30 2024-09-13T16:51:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.908+0000 7f8837200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88380b4be0 con 0x7f88380b4390 2024-09-13T16:51:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.908+0000 7f8837200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88380cf330 con 0x7f883806fb50 2024-09-13T16:51:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.908+0000 7f8835400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f883806ff30 0x7f88380b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8836800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88380b4390 0x7f883815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f883806ff30 0x7f88380b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39054/0 (socket says 172.21.15.3:39054) 2024-09-13T16:51:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835400700 1 -- 172.21.15.3:0/391257401 learned_addr learned my addr 172.21.15.3:0/391257401 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835e00700 1 --2- 172.21.15.3:0/391257401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806fb50 0x7f88380cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835400700 1 -- 172.21.15.3:0/391257401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806fb50 msgr2=0x7f88380cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835400700 1 --2- 172.21.15.3:0/391257401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806fb50 0x7f88380cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835400700 1 -- 172.21.15.3:0/391257401 >> v2:172.21.15.157:3300/0 conn(0x7f88380b4390 msgr2=0x7f883815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835400700 1 --2- 172.21.15.3:0/391257401 >> v2:172.21.15.157:3300/0 conn(0x7f88380b4390 0x7f883815e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835400700 1 -- 172.21.15.3:0/391257401 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88380cf610 con 0x7f883806ff30 2024-09-13T16:51:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8835400700 1 --2- 172.21.15.3:0/391257401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806ff30 0x7f88380b3e50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f88380b28b0 tx=0x7f8824009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8a47b216ac85dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8834a00700 1 -- 172.21.15.3:0/391257401 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824011830 con 0x7f883806ff30 2024-09-13T16:51:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8834a00700 1 -- 172.21.15.3:0/391257401 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8824011990 con 0x7f883806ff30 2024-09-13T16:51:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8834a00700 1 -- 172.21.15.3:0/391257401 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824011d20 con 0x7f883806ff30 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 -- 172.21.15.3:0/391257401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806ff30 msgr2=0x7f88380b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 --2- 172.21.15.3:0/391257401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806ff30 0x7f88380b3e50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f88380b28b0 tx=0x7f8824009bf0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 -- 172.21.15.3:0/391257401 shutdown_connections 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 --2- 172.21.15.3:0/391257401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806ff30 0x7f88380b3e50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 --2- 172.21.15.3:0/391257401 >> v2:172.21.15.157:3300/0 conn(0x7f88380b4390 0x7f883815e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 --2- 172.21.15.3:0/391257401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806fb50 0x7f88380cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 -- 172.21.15.3:0/391257401 >> 172.21.15.3:0/391257401 conn(0x7f88380b2910 msgr2=0x7f88380b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 -- 172.21.15.3:0/391257401 shutdown_connections 2024-09-13T16:51:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.912+0000 7f8837200700 1 -- 172.21.15.3:0/391257401 wait complete. 2024-09-13T16:51:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 Processor -- start 2024-09-13T16:51:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 -- start start 2024-09-13T16:51:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806fb50 0x7f8838146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806ff30 0x7f8838158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88380b4390 0x7f8838141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838158a60 con 0x7f883806fb50 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838147750 con 0x7f88380b4390 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8838146b40 con 0x7f883806ff30 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806fb50 0x7f8838146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806fb50 0x7f8838146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39056/0 (socket says 172.21.15.3:39056) 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 -- 172.21.15.3:0/2835028296 learned_addr learned my addr 172.21.15.3:0/2835028296 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 -- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806ff30 msgr2=0x7f8838158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806ff30 0x7f8838158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 -- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88380b4390 msgr2=0x7f8838141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88380b4390 0x7f8838141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 -- 172.21.15.3:0/2835028296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824009570 con 0x7f883806fb50 2024-09-13T16:51:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8835e00700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806fb50 0x7f8838146560 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f882800e960 tx=0x7f882800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f882e000700 1 -- 172.21.15.3:0/2835028296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800c850 con 0x7f883806fb50 2024-09-13T16:51:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f882e000700 1 -- 172.21.15.3:0/2835028296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882800c9b0 con 0x7f883806fb50 2024-09-13T16:51:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f882e000700 1 -- 172.21.15.3:0/2835028296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800cd40 con 0x7f883806fb50 2024-09-13T16:51:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8838146e20 con 0x7f883806fb50 2024-09-13T16:51:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.916+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88381473e0 con 0x7f883806fb50 2024-09-13T16:51:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.920+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88380cfe10 con 0x7f883806fb50 2024-09-13T16:51:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.920+0000 7f882e000700 1 -- 172.21.15.3:0/2835028296 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8828013470 con 0x7f883806fb50 2024-09-13T16:51:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.920+0000 7f882e000700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8818066730 0x7f8818068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.924+0000 7f882e000700 1 -- 172.21.15.3:0/2835028296 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8828084670 con 0x7f883806fb50 2024-09-13T16:51:49.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.924+0000 7f8835400700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8818066730 0x7f8818068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:49.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.924+0000 7f8835400700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8818066730 0x7f8818068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88381f6160 tx=0x7f8824010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:49.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:49.928+0000 7f882e000700 1 -- 172.21.15.3:0/2835028296 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8828051610 con 0x7f883806fb50 2024-09-13T16:51:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.152+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8838165590 con 0x7f883806fb50 2024-09-13T16:51:50.160 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-13T16:51:50.156+0000 7f9c4ac00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:51:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f882e000700 1 -- 172.21.15.3:0/2835028296 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f882800cea0 con 0x7f883806fb50 2024-09-13T16:51:50.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8818066730 msgr2=0x7f8818068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8818066730 0x7f8818068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88381f6160 tx=0x7f8824010000 comp rx=0 tx=0).stop 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806fb50 msgr2=0x7f8838146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806fb50 0x7f8838146560 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f882800e960 tx=0x7f882800ed20 comp rx=0 tx=0).stop 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 shutdown_connections 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8818066730 0x7f8818068bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88380b4390 0x7f8838141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883806ff30 0x7f8838158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 --2- 172.21.15.3:0/2835028296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f883806fb50 0x7f8838146560 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 >> 172.21.15.3:0/2835028296 conn(0x7f88380b2910 msgr2=0x7f88380b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.168+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 shutdown_connections 2024-09-13T16:51:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.172+0000 7f8837200700 1 -- 172.21.15.3:0/2835028296 wait complete. 2024-09-13T16:51:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T16:51:50.183 INFO:tasks.ceph.mds.d.smithi157.stderr: -5653> 2024-09-13T16:50:31.936+0000 7fcf3bc00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:51:50.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T16:51:50.255 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-13T16:51:50.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.388+0000 7fe6f9000700 1 Processor -- start 2024-09-13T16:51:50.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.388+0000 7fe6f9000700 1 -- start start 2024-09-13T16:51:50.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.388+0000 7fe6f9000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6f40cddb0 0x7fe6f415c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:50.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.388+0000 7fe6f9000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 0x7fe6f40b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:50.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.388+0000 7fe6f9000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f406d5d0 con 0x7fe6f4155250 2024-09-13T16:51:50.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.388+0000 7fe6f9000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f40ce240 con 0x7fe6f40cddb0 2024-09-13T16:51:50.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.388+0000 7fe6f9000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40b5c00 con 0x7fe6f40b5250 2024-09-13T16:51:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.388+0000 7fe6f1400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6f40cddb0 0x7fe6f415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f1400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6f40cddb0 0x7fe6f415c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39062/0 (socket says 172.21.15.3:39062) 2024-09-13T16:51:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f1400700 1 -- 172.21.15.3:0/1472918386 learned_addr learned my addr 172.21.15.3:0/1472918386 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f2800700 1 --2- 172.21.15.3:0/1472918386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 0x7fe6f40b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f1400700 1 -- 172.21.15.3:0/1472918386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 msgr2=0x7fe6f40b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f1400700 1 --2- 172.21.15.3:0/1472918386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 0x7fe6f40b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f1400700 1 -- 172.21.15.3:0/1472918386 >> v1:172.21.15.157:6789/0 conn(0x7fe6f4155250 legacy=0x7fe6f415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f1400700 1 -- 172.21.15.3:0/1472918386 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f415ba50 con 0x7fe6f40cddb0 2024-09-13T16:51:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f1400700 1 --2- 172.21.15.3:0/1472918386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f40cddb0 0x7fe6f415c390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009940 tx=0x7fe6e0009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8097203c8130966f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f0a00700 1 -- 172.21.15.3:0/1472918386 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0011830 con 0x7fe6f40cddb0 2024-09-13T16:51:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f0a00700 1 -- 172.21.15.3:0/1472918386 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e0011990 con 0x7fe6f40cddb0 2024-09-13T16:51:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f0a00700 1 -- 172.21.15.3:0/1472918386 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0011d20 con 0x7fe6f40cddb0 2024-09-13T16:51:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f9000700 1 -- 172.21.15.3:0/1472918386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f40cddb0 msgr2=0x7fe6f415c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f9000700 1 --2- 172.21.15.3:0/1472918386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f40cddb0 0x7fe6f415c390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009940 tx=0x7fe6e0009c50 comp rx=0 tx=0).stop 2024-09-13T16:51:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f9000700 1 -- 172.21.15.3:0/1472918386 shutdown_connections 2024-09-13T16:51:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f9000700 1 --2- 172.21.15.3:0/1472918386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 0x7fe6f40b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f9000700 1 --2- 172.21.15.3:0/1472918386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f40cddb0 0x7fe6f415c390 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f9000700 1 -- 172.21.15.3:0/1472918386 >> 172.21.15.3:0/1472918386 conn(0x7fe6f4071f40 msgr2=0x7fe6f406b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f9000700 1 -- 172.21.15.3:0/1472918386 shutdown_connections 2024-09-13T16:51:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.392+0000 7fe6f9000700 1 -- 172.21.15.3:0/1472918386 wait complete. 2024-09-13T16:51:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f9000700 1 Processor -- start 2024-09-13T16:51:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f9000700 1 -- start start 2024-09-13T16:51:50.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f9000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 0x7fe6f414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:50.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f40cddb0 0x7fe6f40b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:50.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f9000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f4155250 0x7fe6f40b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:50.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f9000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f41f2820 con 0x7fe6f4155250 2024-09-13T16:51:50.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f9000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f414b7c0 con 0x7fe6f40cddb0 2024-09-13T16:51:50.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f9000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40c9ef0 con 0x7fe6f40b5250 2024-09-13T16:51:50.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f4155250 0x7fe6f40b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:50.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f4155250 0x7fe6f40b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39076/0 (socket says 172.21.15.3:39076) 2024-09-13T16:51:50.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 -- 172.21.15.3:0/186587101 learned_addr learned my addr 172.21.15.3:0/186587101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:50.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f1400700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f40cddb0 0x7fe6f40b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:50.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f1e00700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 0x7fe6f414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:50.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 -- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 msgr2=0x7fe6f414abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:50.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 0x7fe6f414abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 -- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f40cddb0 msgr2=0x7fe6f40b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:50.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f40cddb0 0x7fe6f40b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:50.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 -- 172.21.15.3:0/186587101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0009570 con 0x7fe6f4155250 2024-09-13T16:51:50.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.396+0000 7fe6f2800700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f4155250 0x7fe6f40b4850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00efc0 tx=0x7fe6ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:50.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.400+0000 7fe6ea000700 1 -- 172.21.15.3:0/186587101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec009ea0 con 0x7fe6f4155250 2024-09-13T16:51:50.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.400+0000 7fe6ea000700 1 -- 172.21.15.3:0/186587101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec00f040 con 0x7fe6f4155250 2024-09-13T16:51:50.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.400+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f41f2a50 con 0x7fe6f4155250 2024-09-13T16:51:50.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.400+0000 7fe6ea000700 1 -- 172.21.15.3:0/186587101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec00ced0 con 0x7fe6f4155250 2024-09-13T16:51:50.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.400+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f41f3010 con 0x7fe6f4155250 2024-09-13T16:51:50.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.400+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f406d800 con 0x7fe6f4155250 2024-09-13T16:51:50.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.400+0000 7fe6ea000700 1 -- 172.21.15.3:0/186587101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6ec014480 con 0x7fe6f4155250 2024-09-13T16:51:50.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.404+0000 7fe6ea000700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6d4066730 0x7fe6d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:50.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.404+0000 7fe6ea000700 1 -- 172.21.15.3:0/186587101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fe6ec084dc0 con 0x7fe6f4155250 2024-09-13T16:51:50.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.404+0000 7fe6f1e00700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6d4066730 0x7fe6d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:50.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.408+0000 7fe6f1e00700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6d4066730 0x7fe6d4068bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4000f60 tx=0x7fe6e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:50.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.412+0000 7fe6ea000700 1 -- 172.21.15.3:0/186587101 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6f406d800 con 0x7fe6f4155250 2024-09-13T16:51:50.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:50.628+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe6f406d800 con 0x7fe6f4155250 2024-09-13T16:51:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.172+0000 7fe6ea000700 1 -- 172.21.15.3:0/186587101 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe6f406d800 con 0x7fe6f4155250 2024-09-13T16:51:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.176+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6d4066730 msgr2=0x7fe6d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.176+0000 7fe6f9000700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6d4066730 0x7fe6d4068bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4000f60 tx=0x7fe6e4009040 comp rx=0 tx=0).stop 2024-09-13T16:51:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.176+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f4155250 msgr2=0x7fe6f40b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.176+0000 7fe6f9000700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f4155250 0x7fe6f40b4850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00efc0 tx=0x7fe6ec00c3a0 comp rx=0 tx=0).stop 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.180+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 shutdown_connections 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.180+0000 7fe6f9000700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe6d4066730 0x7fe6d4068bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.180+0000 7fe6f9000700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6f4155250 0x7fe6f40b4850 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.180+0000 7fe6f9000700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f40cddb0 0x7fe6f40b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.180+0000 7fe6f9000700 1 --2- 172.21.15.3:0/186587101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f40b5250 0x7fe6f414abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.180+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 >> 172.21.15.3:0/186587101 conn(0x7fe6f4071f40 msgr2=0x7fe6f41598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.180+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 shutdown_connections 2024-09-13T16:51:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.180+0000 7fe6f9000700 1 -- 172.21.15.3:0/186587101 wait complete. 2024-09-13T16:51:51.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T16:51:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.384+0000 7fb893000700 1 Processor -- start 2024-09-13T16:51:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.384+0000 7fb893000700 1 -- start start 2024-09-13T16:51:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb893000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb88c15aa00 0x7fb88c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb893000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c0cc120 con 0x7fb88c06aaa0 2024-09-13T16:51:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb893000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c0cc370 con 0x7fb88c0b30b0 2024-09-13T16:51:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb893000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c0cc650 con 0x7fb88c15aa00 2024-09-13T16:51:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb88be00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb88c15aa00 0x7fb88c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb891000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb88c06aaa0 0x7fb88c15e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47356/0 (socket says 172.21.15.3:47356) 2024-09-13T16:51:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb891000700 1 -- 172.21.15.3:0/2716859394 learned_addr learned my addr 172.21.15.3:0/2716859394 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb88be00700 1 -- 172.21.15.3:0/2716859394 >> v1:172.21.15.157:6789/0 conn(0x7fb88c0b30b0 legacy=0x7fb88c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb88be00700 1 -- 172.21.15.3:0/2716859394 >> v1:172.21.15.3:6789/0 conn(0x7fb88c06aaa0 legacy=0x7fb88c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb88be00700 1 -- 172.21.15.3:0/2716859394 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0cc930 con 0x7fb88c15aa00 2024-09-13T16:51:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.388+0000 7fb88aa00700 1 -- 172.21.15.3:0/2716859394 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672455755 0 0) 0x7fb88c0cc120 con 0x7fb88c06aaa0 2024-09-13T16:51:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb88be00700 1 --2- 172.21.15.3:0/2716859394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cf640 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009850 tx=0x7fb87c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad17970f1b211613 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb88aa00700 1 -- 172.21.15.3:0/2716859394 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c011830 con 0x7fb88c15aa00 2024-09-13T16:51:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb88aa00700 1 -- 172.21.15.3:0/2716859394 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c011990 con 0x7fb88c15aa00 2024-09-13T16:51:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb88aa00700 1 -- 172.21.15.3:0/2716859394 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c011d20 con 0x7fb88c15aa00 2024-09-13T16:51:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb893000700 1 -- 172.21.15.3:0/2716859394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c15aa00 msgr2=0x7fb88c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb893000700 1 --2- 172.21.15.3:0/2716859394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cf640 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009850 tx=0x7fb87c009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb893000700 1 -- 172.21.15.3:0/2716859394 shutdown_connections 2024-09-13T16:51:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb893000700 1 --2- 172.21.15.3:0/2716859394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cf640 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb893000700 1 -- 172.21.15.3:0/2716859394 >> 172.21.15.3:0/2716859394 conn(0x7fb88c06b4a0 msgr2=0x7fb88c06b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb893000700 1 -- 172.21.15.3:0/2716859394 shutdown_connections 2024-09-13T16:51:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.392+0000 7fb893000700 1 -- 172.21.15.3:0/2716859394 wait complete. 2024-09-13T16:51:51.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb893000700 1 Processor -- start 2024-09-13T16:51:51.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb893000700 1 -- start start 2024-09-13T16:51:51.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb893000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb88c06aaa0 0x7fb88c148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb893000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c0b30b0 0x7fb88c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb893000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb893000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c151f70 con 0x7fb88c15aa00 2024-09-13T16:51:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb893000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c151ae0 con 0x7fb88c0b30b0 2024-09-13T16:51:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb893000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c150ed0 con 0x7fb88c06aaa0 2024-09-13T16:51:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb891000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb891000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39092/0 (socket says 172.21.15.3:39092) 2024-09-13T16:51:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.396+0000 7fb891000700 1 -- 172.21.15.3:0/3066376643 learned_addr learned my addr 172.21.15.3:0/3066376643 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb88be00700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb88c06aaa0 0x7fb88c148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb88b400700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c0b30b0 0x7fb88c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb891000700 1 -- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb88c06aaa0 msgr2=0x7fb88c148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb891000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb88c06aaa0 0x7fb88c148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb891000700 1 -- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c0b30b0 msgr2=0x7fb88c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb891000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c0b30b0 0x7fb88c0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb891000700 1 -- 172.21.15.3:0/3066376643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0094b0 con 0x7fb88c15aa00 2024-09-13T16:51:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb88b400700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c0b30b0 0x7fb88c0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb891000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cbb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb88400cbd0 tx=0x7fb88400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb883e00700 1 -- 172.21.15.3:0/3066376643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400aac0 con 0x7fb88c15aa00 2024-09-13T16:51:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb883e00700 1 -- 172.21.15.3:0/3066376643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88400ac20 con 0x7fb88c15aa00 2024-09-13T16:51:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb883e00700 1 -- 172.21.15.3:0/3066376643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8840070d0 con 0x7fb88c15aa00 2024-09-13T16:51:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c1511b0 con 0x7fb88c15aa00 2024-09-13T16:51:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c151770 con 0x7fb88c15aa00 2024-09-13T16:51:51.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.400+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c0b43f0 con 0x7fb88c15aa00 2024-09-13T16:51:51.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.404+0000 7fb883e00700 1 -- 172.21.15.3:0/3066376643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb884007230 con 0x7fb88c15aa00 2024-09-13T16:51:51.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.404+0000 7fb883e00700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb86c066730 0x7fb86c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:51.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.404+0000 7fb883e00700 1 -- 172.21.15.3:0/3066376643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fb8840840f0 con 0x7fb88c15aa00 2024-09-13T16:51:51.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.408+0000 7fb88be00700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb86c066730 0x7fb86c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:51.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.408+0000 7fb88be00700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb86c066730 0x7fb86c068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0090b0 tx=0x7fb87c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:51.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.412+0000 7fb883e00700 1 -- 172.21.15.3:0/3066376643 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb884050f30 con 0x7fb88c15aa00 2024-09-13T16:51:51.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:51.596+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb88c151c20 con 0x7fb88c15aa00 2024-09-13T16:51:52.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.488+0000 7fb883e00700 1 -- 172.21.15.3:0/3066376643 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb884054550 con 0x7fb88c15aa00 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.492+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb86c066730 msgr2=0x7fb86c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.492+0000 7fb893000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb86c066730 0x7fb86c068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0090b0 tx=0x7fb87c008030 comp rx=0 tx=0).stop 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb88c15aa00 msgr2=0x7fb88c0cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cbb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb88400cbd0 tx=0x7fb88400cee0 comp rx=0 tx=0).stop 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 shutdown_connections 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb86c066730 0x7fb86c068bf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb88c15aa00 0x7fb88c0cbb20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb88c0b30b0 0x7fb88c0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 --2- 172.21.15.3:0/3066376643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb88c06aaa0 0x7fb88c148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 >> 172.21.15.3:0/3066376643 conn(0x7fb88c06b4a0 msgr2=0x7fb88c0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 shutdown_connections 2024-09-13T16:51:52.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.496+0000 7fb893000700 1 -- 172.21.15.3:0/3066376643 wait complete. 2024-09-13T16:51:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-13T16:51:52.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T16:51:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.700+0000 7fb423200700 1 Processor -- start 2024-09-13T16:51:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.700+0000 7fb423200700 1 -- start start 2024-09-13T16:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.704+0000 7fb423200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4240717f0 0x7fb4240b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.704+0000 7fb423200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4240b49a0 con 0x7fb4240b4550 2024-09-13T16:51:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.704+0000 7fb423200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb424070750 con 0x7fb42415a8f0 2024-09-13T16:51:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.704+0000 7fb423200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb424070a30 con 0x7fb4240717f0 2024-09-13T16:51:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.704+0000 7fb422800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb4240b4550 0x7fb42415acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47370/0 (socket says 172.21.15.3:47370) 2024-09-13T16:51:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb422800700 1 -- 172.21.15.3:0/2684627275 learned_addr learned my addr 172.21.15.3:0/2684627275 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb421400700 1 --2- 172.21.15.3:0/2684627275 >> v2:172.21.15.157:3300/0 conn(0x7fb4240717f0 0x7fb4240b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb421400700 1 -- 172.21.15.3:0/2684627275 >> v1:172.21.15.157:6789/0 conn(0x7fb42415a8f0 legacy=0x7fb4240cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb421400700 1 -- 172.21.15.3:0/2684627275 >> v1:172.21.15.3:6789/0 conn(0x7fb4240b4550 legacy=0x7fb42415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb421400700 1 -- 172.21.15.3:0/2684627275 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb424070d10 con 0x7fb4240717f0 2024-09-13T16:51:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb421400700 1 --2- 172.21.15.3:0/2684627275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4240717f0 0x7fb4240b4010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb410009850 tx=0x7fb410009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa0232a364902058 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb420a00700 1 -- 172.21.15.3:0/2684627275 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410011830 con 0x7fb4240717f0 2024-09-13T16:51:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb420a00700 1 -- 172.21.15.3:0/2684627275 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410011990 con 0x7fb4240717f0 2024-09-13T16:51:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb420a00700 1 -- 172.21.15.3:0/2684627275 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410011d20 con 0x7fb4240717f0 2024-09-13T16:51:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb423200700 1 -- 172.21.15.3:0/2684627275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4240717f0 msgr2=0x7fb4240b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb423200700 1 --2- 172.21.15.3:0/2684627275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4240717f0 0x7fb4240b4010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb410009850 tx=0x7fb410009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb423200700 1 -- 172.21.15.3:0/2684627275 shutdown_connections 2024-09-13T16:51:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb423200700 1 --2- 172.21.15.3:0/2684627275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4240717f0 0x7fb4240b4010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.708+0000 7fb423200700 1 -- 172.21.15.3:0/2684627275 >> 172.21.15.3:0/2684627275 conn(0x7fb42406b1e0 msgr2=0x7fb42406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 -- 172.21.15.3:0/2684627275 shutdown_connections 2024-09-13T16:51:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 -- 172.21.15.3:0/2684627275 wait complete. 2024-09-13T16:51:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 Processor -- start 2024-09-13T16:51:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 -- start start 2024-09-13T16:51:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4240717f0 0x7fb4241501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4240b4550 0x7fb4241459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb42415a8f0 0x7fb4241517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42414ee70 con 0x7fb4240717f0 2024-09-13T16:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42414f150 con 0x7fb42415a8f0 2024-09-13T16:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.712+0000 7fb423200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42414f430 con 0x7fb4240b4550 2024-09-13T16:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4240717f0 0x7fb4241501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4240717f0 0x7fb4241501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39096/0 (socket says 172.21.15.3:39096) 2024-09-13T16:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 -- 172.21.15.3:0/3936623381 learned_addr learned my addr 172.21.15.3:0/3936623381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb422800700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb42415a8f0 0x7fb4241517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421400700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4240b4550 0x7fb4241459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 -- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4240b4550 msgr2=0x7fb4241459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4240b4550 0x7fb4241459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 -- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb42415a8f0 msgr2=0x7fb4241517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb42415a8f0 0x7fb4241517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 -- 172.21.15.3:0/3936623381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100094b0 con 0x7fb4240717f0 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb421e00700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4240717f0 0x7fb4241501f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c500 tx=0x7fb41400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb41a000700 1 -- 172.21.15.3:0/3936623381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140052d0 con 0x7fb4240717f0 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb41a000700 1 -- 172.21.15.3:0/3936623381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414005430 con 0x7fb4240717f0 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb41a000700 1 -- 172.21.15.3:0/3936623381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140055a0 con 0x7fb4240717f0 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42414f710 con 0x7fb4240717f0 2024-09-13T16:51:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.716+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42414fcd0 con 0x7fb4240717f0 2024-09-13T16:51:52.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.720+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb424067790 con 0x7fb4240717f0 2024-09-13T16:51:52.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.720+0000 7fb41a000700 1 -- 172.21.15.3:0/3936623381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4140038e0 con 0x7fb4240717f0 2024-09-13T16:51:52.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.720+0000 7fb41a000700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb404066760 0x7fb404068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:52.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.720+0000 7fb41a000700 1 -- 172.21.15.3:0/3936623381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fb414083800 con 0x7fb4240717f0 2024-09-13T16:51:52.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.720+0000 7fb421400700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb404066760 0x7fb404068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:52.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.724+0000 7fb421400700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb404066760 0x7fb404068c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb410014040 tx=0x7fb41001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.728+0000 7fb41a000700 1 -- 172.21.15.3:0/3936623381 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb414050870 con 0x7fb4240717f0 2024-09-13T16:51:52.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:52.912+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb42414fef0 con 0x7fb4240717f0 2024-09-13T16:51:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.508+0000 7fb41a000700 1 -- 172.21.15.3:0/3936623381 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb414053e90 con 0x7fb4240717f0 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb404066760 msgr2=0x7fb404068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb404066760 0x7fb404068c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb410014040 tx=0x7fb41001c040 comp rx=0 tx=0).stop 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4240717f0 msgr2=0x7fb4241501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4240717f0 0x7fb4241501f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c500 tx=0x7fb41400c810 comp rx=0 tx=0).stop 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 shutdown_connections 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb404066760 0x7fb404068c20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb42415a8f0 0x7fb4241517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4240b4550 0x7fb4241459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 --2- 172.21.15.3:0/3936623381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4240717f0 0x7fb4241501f0 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c500 tx=0x7fb41400c810 comp rx=0 tx=0).stop 2024-09-13T16:51:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 >> 172.21.15.3:0/3936623381 conn(0x7fb42406b1e0 msgr2=0x7fb4240b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 shutdown_connections 2024-09-13T16:51:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.512+0000 7fb423200700 1 -- 172.21.15.3:0/3936623381 wait complete. 2024-09-13T16:51:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-13T16:51:53.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-13T16:51:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.712+0000 7ff83b200700 1 Processor -- start 2024-09-13T16:51:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.712+0000 7ff83b200700 1 -- start start 2024-09-13T16:51:53.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff83b200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff83c154ff0 0x7ff83c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:53.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff83b200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff83c15b770 0x7ff83c0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:53.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff83b200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83c0d06d0 con 0x7ff83c0cd520 2024-09-13T16:51:53.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff83b200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83c0cdef0 con 0x7ff83c154ff0 2024-09-13T16:51:53.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff83b200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83c15bc40 con 0x7ff83c15b770 2024-09-13T16:51:53.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff839e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff83c154ff0 0x7ff83c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:53.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff839400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff83c0cd520 0x7ff83c15b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37946/0 (socket says 172.21.15.3:37946) 2024-09-13T16:51:53.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff839400700 1 -- 172.21.15.3:0/3694390749 learned_addr learned my addr 172.21.15.3:0/3694390749 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:53.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.716+0000 7ff83a800700 1 --2- 172.21.15.3:0/3694390749 >> v2:172.21.15.157:3300/0 conn(0x7ff83c15b770 0x7ff83c0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:53.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff839e00700 1 -- 172.21.15.3:0/3694390749 >> v2:172.21.15.157:3300/0 conn(0x7ff83c15b770 msgr2=0x7ff83c0cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff839e00700 1 --2- 172.21.15.3:0/3694390749 >> v2:172.21.15.157:3300/0 conn(0x7ff83c15b770 0x7ff83c0cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff839e00700 1 -- 172.21.15.3:0/3694390749 >> v1:172.21.15.3:6789/0 conn(0x7ff83c0cd520 legacy=0x7ff83c15b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff838a00700 1 -- 172.21.15.3:0/3694390749 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058219328 0 0) 0x7ff83c0d06d0 con 0x7ff83c0cd520 2024-09-13T16:51:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff839e00700 1 -- 172.21.15.3:0/3694390749 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c15bf20 con 0x7ff83c154ff0 2024-09-13T16:51:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff839e00700 1 --2- 172.21.15.3:0/3694390749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15c220 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0cf840 tx=0x7ff82c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97ee8ee2fe70e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:53.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff838a00700 1 -- 172.21.15.3:0/3694390749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c011830 con 0x7ff83c154ff0 2024-09-13T16:51:53.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff838a00700 1 -- 172.21.15.3:0/3694390749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c011990 con 0x7ff83c154ff0 2024-09-13T16:51:53.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff838a00700 1 -- 172.21.15.3:0/3694390749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c011d20 con 0x7ff83c154ff0 2024-09-13T16:51:53.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff83b200700 1 -- 172.21.15.3:0/3694390749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 msgr2=0x7ff83c15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:53.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff83b200700 1 --2- 172.21.15.3:0/3694390749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15c220 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0cf840 tx=0x7ff82c009b30 comp rx=0 tx=0).stop 2024-09-13T16:51:53.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff83b200700 1 -- 172.21.15.3:0/3694390749 shutdown_connections 2024-09-13T16:51:53.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff83b200700 1 --2- 172.21.15.3:0/3694390749 >> v2:172.21.15.157:3300/0 conn(0x7ff83c15b770 0x7ff83c0cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:53.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff83b200700 1 --2- 172.21.15.3:0/3694390749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15c220 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:53.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff83b200700 1 -- 172.21.15.3:0/3694390749 >> 172.21.15.3:0/3694390749 conn(0x7ff83c0b6210 msgr2=0x7ff83c0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:53.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff83b200700 1 -- 172.21.15.3:0/3694390749 shutdown_connections 2024-09-13T16:51:53.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.720+0000 7ff83b200700 1 -- 172.21.15.3:0/3694390749 wait complete. 2024-09-13T16:51:53.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.724+0000 7ff83b200700 1 Processor -- start 2024-09-13T16:51:53.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.724+0000 7ff83b200700 1 -- start start 2024-09-13T16:51:53.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83b200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0cd520 0x7ff83c13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:53.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83b200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:53.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c15b770 0x7ff83c15d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83b200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c13fd90 con 0x7ff83c154ff0 2024-09-13T16:51:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c1401b0 con 0x7ff83c15b770 2024-09-13T16:51:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83b200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c158950 con 0x7ff83c0cd520 2024-09-13T16:51:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43246/0 (socket says 172.21.15.3:43246) 2024-09-13T16:51:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 -- 172.21.15.3:0/263732411 learned_addr learned my addr 172.21.15.3:0/263732411 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839e00700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0cd520 0x7ff83c13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83a800700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c15b770 0x7ff83c15d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 -- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0cd520 msgr2=0x7ff83c13eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0cd520 0x7ff83c13eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 -- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c15b770 msgr2=0x7ff83c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c15b770 0x7ff83c15d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 -- 172.21.15.3:0/263732411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0094b0 con 0x7ff83c154ff0 2024-09-13T16:51:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff839400700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15caf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c580 tx=0x7ff82800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff832000700 1 -- 172.21.15.3:0/263732411 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828004ca0 con 0x7ff83c154ff0 2024-09-13T16:51:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff832000700 1 -- 172.21.15.3:0/263732411 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828004e20 con 0x7ff83c154ff0 2024-09-13T16:51:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff832000700 1 -- 172.21.15.3:0/263732411 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828007310 con 0x7ff83c154ff0 2024-09-13T16:51:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c150b10 con 0x7ff83c154ff0 2024-09-13T16:51:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.728+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c13f6d0 con 0x7ff83c154ff0 2024-09-13T16:51:53.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.732+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83c13fed0 con 0x7ff83c154ff0 2024-09-13T16:51:53.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.732+0000 7ff832000700 1 -- 172.21.15.3:0/263732411 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff828007470 con 0x7ff83c154ff0 2024-09-13T16:51:53.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.736+0000 7ff832000700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff81c066760 0x7ff81c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:53.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.736+0000 7ff832000700 1 -- 172.21.15.3:0/263732411 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7ff828083d20 con 0x7ff83c154ff0 2024-09-13T16:51:53.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.736+0000 7ff839e00700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff81c066760 0x7ff81c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.736+0000 7ff839e00700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff81c066760 0x7ff81c068c20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000c00 tx=0x7ff82c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.740+0000 7ff832000700 1 -- 172.21.15.3:0/263732411 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff83c13fed0 con 0x7ff83c154ff0 2024-09-13T16:51:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:53.928+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff83c13fed0 con 0x7ff83c154ff0 2024-09-13T16:51:54.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.516+0000 7ff832000700 1 -- 172.21.15.3:0/263732411 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff83c13fed0 con 0x7ff83c154ff0 2024-09-13T16:51:54.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.520+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff81c066760 msgr2=0x7ff81c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:54.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.520+0000 7ff83b200700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff81c066760 0x7ff81c068c20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000c00 tx=0x7ff82c008030 comp rx=0 tx=0).stop 2024-09-13T16:51:54.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.520+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 msgr2=0x7ff83c15caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:54.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.520+0000 7ff83b200700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15caf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c580 tx=0x7ff82800c890 comp rx=0 tx=0).stop 2024-09-13T16:51:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.524+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 shutdown_connections 2024-09-13T16:51:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.524+0000 7ff83b200700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff81c066760 0x7ff81c068c20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.524+0000 7ff83b200700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c15b770 0x7ff83c15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.524+0000 7ff83b200700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c154ff0 0x7ff83c15caf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.524+0000 7ff83b200700 1 --2- 172.21.15.3:0/263732411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0cd520 0x7ff83c13eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.524+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 >> 172.21.15.3:0/263732411 conn(0x7ff83c0b6210 msgr2=0x7ff83c0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.524+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 shutdown_connections 2024-09-13T16:51:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.524+0000 7ff83b200700 1 -- 172.21.15.3:0/263732411 wait complete. 2024-09-13T16:51:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' removed 2024-09-13T16:51:54.557 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:51:54.557 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T16:51:54.558 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:51:54.558 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:51:54.558 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T16:51:54.558 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T16:51:54.558 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T16:51:54.558 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:51:54.558 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:51:54.558 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T16:51:54.559 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:51:54.559 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:51:54.559 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:51:54.559 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T16:51:54.559 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:51:54.559 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:51:54.581 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:51:54.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:51:54.585 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:51:54.585 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:51:54.586 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T16:51:54.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T16:51:54.587 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:51:54.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:51:54.632 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:51:54.633 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:54.636 INFO:tasks.ceph.mds.c:Started 2024-09-13T16:51:54.636 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:54.638 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:51:54.638 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:54.640 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:51:54.640 DEBUG:teuthology.parallel:result is None 2024-09-13T16:51:54.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T16:51:54.750 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-13T16:51:54.754 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-13T16:51:54.759 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-13T16:51:54.772 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-13T16:51:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c2e800700 1 Processor -- start 2024-09-13T16:51:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c2e800700 1 -- start start 2024-09-13T16:51:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c2e800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3c28151c90 0x7f3c2806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c2e800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c280cfb80 0x7f3c280cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c2e800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c28152070 con 0x7f3c280cf740 2024-09-13T16:51:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c2e800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2815e7a0 con 0x7f3c28151c90 2024-09-13T16:51:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c2e800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2815ea80 con 0x7f3c280cfb80 2024-09-13T16:51:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c26a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3c280cf740 0x7f3c280b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37958/0 (socket says 172.21.15.3:37958) 2024-09-13T16:51:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c26a00700 1 -- 172.21.15.3:0/1125360461 learned_addr learned my addr 172.21.15.3:0/1125360461 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c27400700 1 --2- 172.21.15.3:0/1125360461 >> v2:172.21.15.3:3300/0 conn(0x7f3c28151c90 0x7f3c2806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c27e00700 1 --2- 172.21.15.3:0/1125360461 >> v2:172.21.15.157:3300/0 conn(0x7f3c280cfb80 0x7f3c280cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c26000700 1 -- 172.21.15.3:0/1125360461 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195309258 0 0) 0x7f3c28152070 con 0x7f3c280cf740 2024-09-13T16:51:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.804+0000 7f3c26000700 1 -- 172.21.15.3:0/1125360461 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c0c0035d0 con 0x7f3c280cf740 2024-09-13T16:51:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c27400700 1 -- 172.21.15.3:0/1125360461 >> v2:172.21.15.157:3300/0 conn(0x7f3c280cfb80 msgr2=0x7f3c280cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c27400700 1 --2- 172.21.15.3:0/1125360461 >> v2:172.21.15.157:3300/0 conn(0x7f3c280cfb80 0x7f3c280cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c27400700 1 -- 172.21.15.3:0/1125360461 >> v1:172.21.15.3:6789/0 conn(0x7f3c280cf740 legacy=0x7f3c280b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c27400700 1 -- 172.21.15.3:0/1125360461 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2815ed60 con 0x7f3c28151c90 2024-09-13T16:51:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c27400700 1 --2- 172.21.15.3:0/1125360461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2806b7c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c28155770 tx=0x7f3c18009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4363f04b9edb640c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:54.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c26000700 1 -- 172.21.15.3:0/1125360461 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18011830 con 0x7f3c28151c90 2024-09-13T16:51:54.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c26000700 1 -- 172.21.15.3:0/1125360461 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c18011990 con 0x7f3c28151c90 2024-09-13T16:51:54.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c26000700 1 -- 172.21.15.3:0/1125360461 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18011d20 con 0x7f3c28151c90 2024-09-13T16:51:54.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c2e800700 1 -- 172.21.15.3:0/1125360461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 msgr2=0x7f3c2806b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:54.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c2e800700 1 --2- 172.21.15.3:0/1125360461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2806b7c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c28155770 tx=0x7f3c18009b30 comp rx=0 tx=0).stop 2024-09-13T16:51:54.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c2e800700 1 -- 172.21.15.3:0/1125360461 shutdown_connections 2024-09-13T16:51:54.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c2e800700 1 --2- 172.21.15.3:0/1125360461 >> v2:172.21.15.157:3300/0 conn(0x7f3c280cfb80 0x7f3c280cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c2e800700 1 --2- 172.21.15.3:0/1125360461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2806b7c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c2e800700 1 -- 172.21.15.3:0/1125360461 >> 172.21.15.3:0/1125360461 conn(0x7f3c2806d3c0 msgr2=0x7f3c2806d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:54.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c2e800700 1 -- 172.21.15.3:0/1125360461 shutdown_connections 2024-09-13T16:51:54.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.808+0000 7f3c2e800700 1 -- 172.21.15.3:0/1125360461 wait complete. 2024-09-13T16:51:54.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c2e800700 1 Processor -- start 2024-09-13T16:51:54.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c2e800700 1 -- start start 2024-09-13T16:51:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c2e800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c280cf740 0x7f3c281498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c2e800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c2e800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2815ae50 0x7f3c280b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c2e800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2815b300 con 0x7f3c28151c90 2024-09-13T16:51:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c2e800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2815c2e0 con 0x7f3c2815ae50 2024-09-13T16:51:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c2e800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2815ce60 con 0x7f3c280cf740 2024-09-13T16:51:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c26a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c26a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2815a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43290/0 (socket says 172.21.15.3:43290) 2024-09-13T16:51:54.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c26a00700 1 -- 172.21.15.3:0/2975072933 learned_addr learned my addr 172.21.15.3:0/2975072933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:54.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c27e00700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2815ae50 0x7f3c280b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:54.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.812+0000 7f3c27400700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c280cf740 0x7f3c281498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:54.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c26a00700 1 -- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c280cf740 msgr2=0x7f3c281498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:54.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c26a00700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c280cf740 0x7f3c281498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c26a00700 1 -- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2815ae50 msgr2=0x7f3c280b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:54.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c26a00700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2815ae50 0x7f3c280b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:54.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c26a00700 1 -- 172.21.15.3:0/2975072933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180094b0 con 0x7f3c28151c90 2024-09-13T16:51:54.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c26a00700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2815a910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400c4d0 tx=0x7f3c1400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:54.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c1f400700 1 -- 172.21.15.3:0/2975072933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14005d90 con 0x7f3c28151c90 2024-09-13T16:51:54.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c1f400700 1 -- 172.21.15.3:0/2975072933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c140070d0 con 0x7f3c28151c90 2024-09-13T16:51:54.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c281557a0 con 0x7f3c28151c90 2024-09-13T16:51:54.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c1f400700 1 -- 172.21.15.3:0/2975072933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14007480 con 0x7f3c28151c90 2024-09-13T16:51:54.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2815c810 con 0x7f3c28151c90 2024-09-13T16:51:54.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2815cfd0 con 0x7f3c28151c90 2024-09-13T16:51:54.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.820+0000 7f3c1f400700 1 -- 172.21.15.3:0/2975072933 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c140075e0 con 0x7f3c28151c90 2024-09-13T16:51:54.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.820+0000 7f3c1f400700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3c08066730 0x7f3c08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:54.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.820+0000 7f3c1f400700 1 -- 172.21.15.3:0/2975072933 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3c14083440 con 0x7f3c28151c90 2024-09-13T16:51:54.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.820+0000 7f3c27400700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3c08066730 0x7f3c08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:54.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.824+0000 7f3c27400700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3c08066730 0x7f3c08068bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c18008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:54.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:54.828+0000 7f3c1f400700 1 -- 172.21.15.3:0/2975072933 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c140508d0 con 0x7f3c28151c90 2024-09-13T16:51:55.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.024+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3c2806b210 con 0x7f3c28151c90 2024-09-13T16:51:55.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.812+0000 7f3c1f400700 1 -- 172.21.15.3:0/2975072933 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3c14053ef0 con 0x7f3c28151c90 2024-09-13T16:51:55.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3c08066730 msgr2=0x7f3c08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3c08066730 0x7f3c08068bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c18008030 comp rx=0 tx=0).stop 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 msgr2=0x7f3c2815a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2815a910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400c4d0 tx=0x7f3c1400c7e0 comp rx=0 tx=0).stop 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 shutdown_connections 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3c08066730 0x7f3c08068bf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2815ae50 0x7f3c280b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c28151c90 0x7f3c2815a910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 --2- 172.21.15.3:0/2975072933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c280cf740 0x7f3c281498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:55.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 >> 172.21.15.3:0/2975072933 conn(0x7f3c2806d3c0 msgr2=0x7f3c28158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:55.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 shutdown_connections 2024-09-13T16:51:55.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:55.816+0000 7f3c2e800700 1 -- 172.21.15.3:0/2975072933 wait complete. 2024-09-13T16:51:55.823 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-13T16:51:55.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T16:51:56.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f71a1200700 1 Processor -- start 2024-09-13T16:51:56.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f71a1200700 1 -- start start 2024-09-13T16:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f71a1200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f719c06fff0 0x7f719c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f71a1200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c15d4d0 0x7f719c15d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f71a1200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c1617e0 con 0x7f719c0b3f40 2024-09-13T16:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f71a1200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719c15e2a0 con 0x7f719c06fff0 2024-09-13T16:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f71a1200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c160a20 con 0x7f719c15d4d0 2024-09-13T16:51:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719b400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f719c0b3f40 0x7f719c06f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37972/0 (socket says 172.21.15.3:37972) 2024-09-13T16:51:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719b400700 1 -- 172.21.15.3:0/3850630478 learned_addr learned my addr 172.21.15.3:0/3850630478 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719be00700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c15d4d0 0x7f719c15d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014755381 0 0) 0x7f719c1617e0 con 0x7f719c0b3f40 2024-09-13T16:51:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71800035d0 con 0x7f719c0b3f40 2024-09-13T16:51:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f7192a00700 1 --2- 172.21.15.3:0/3850630478 >> v2:172.21.15.157:3300/0 conn(0x7f719c06fff0 0x7f719c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 820194838 0 0) 0x7f71800035d0 con 0x7f719c0b3f40 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c15d4d0 msgr2=0x7f719c15d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c15d4d0 0x7f719c15d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 >> v2:172.21.15.157:3300/0 conn(0x7f719c06fff0 msgr2=0x7f719c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 --2- 172.21.15.3:0/3850630478 >> v2:172.21.15.157:3300/0 conn(0x7f719c06fff0 0x7f719c0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c160d00 con 0x7f719c0b3f40 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.020+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f718c001690 con 0x7f719c0b3f40 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 >> v1:172.21.15.3:6789/0 conn(0x7f719c0b3f40 legacy=0x7f719c06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7180003a20 0x7f7180003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7180004870 0x7f7180004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180005230 0x7f718000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c15e2a0 con 0x7f7180003a20 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c160a20 con 0x7f7180005230 2024-09-13T16:51:56.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718000c070 con 0x7f7180004870 2024-09-13T16:51:56.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719be00700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c15d4d0 0x7f719c15d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:56.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719b400700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7180003a20 0x7f7180003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719be00700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180005230 0x7f718000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f7192a00700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7180004870 0x7f7180004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719b400700 1 -- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7180004870 msgr2=0x7f7180004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:56.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719b400700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7180004870 0x7f7180004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719b400700 1 -- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180005230 msgr2=0x7f718000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:56.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719b400700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7180005230 0x7f718000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719b400700 1 -- 172.21.15.3:0/3850630478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800035d0 con 0x7f7180003a20 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719b400700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7180003a20 0x7f7180003e40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c2a0 tx=0x7f71800058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b98ad605ade955 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c007c80 con 0x7f7180003a20 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c00ce00 con 0x7f7180003a20 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719aa00700 1 -- 172.21.15.3:0/3850630478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c005bc0 con 0x7f7180003a20 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f71a1200700 1 -- 172.21.15.3:0/3850630478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7180003a20 msgr2=0x7f7180003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f71a1200700 1 --2- 172.21.15.3:0/3850630478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7180003a20 0x7f7180003e40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c2a0 tx=0x7f71800058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f719be00700 1 -- 172.21.15.3:0/3850630478 reap_dead start 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f71a1200700 1 -- 172.21.15.3:0/3850630478 shutdown_connections 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f71a1200700 1 -- 172.21.15.3:0/3850630478 >> 172.21.15.3:0/3850630478 conn(0x7f719c0b2910 msgr2=0x7f719c0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f71a1200700 1 -- 172.21.15.3:0/3850630478 shutdown_connections 2024-09-13T16:51:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.024+0000 7f71a1200700 1 -- 172.21.15.3:0/3850630478 wait complete. 2024-09-13T16:51:56.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.028+0000 7f71a1200700 1 Processor -- start 2024-09-13T16:51:56.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.028+0000 7f71a1200700 1 -- start start 2024-09-13T16:51:56.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.028+0000 7f71a1200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c13f700 0x7f719c1f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.028+0000 7f71a1200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f719c145bc0 0x7f719c1f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.028+0000 7f71a1200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f719c06b4a0 0x7f719c06b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.028+0000 7f71a1200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1f6e70 con 0x7f719c145bc0 2024-09-13T16:51:56.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.028+0000 7f71a1200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c06be60 con 0x7f719c06b4a0 2024-09-13T16:51:56.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.028+0000 7f71a1200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c13fae0 con 0x7f719c13f700 2024-09-13T16:51:56.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f719c145bc0 0x7f719c1f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f719c145bc0 0x7f719c1f2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43296/0 (socket says 172.21.15.3:43296) 2024-09-13T16:51:56.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 -- 172.21.15.3:0/4210750548 learned_addr learned my addr 172.21.15.3:0/4210750548 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:56.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f719b400700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c13f700 0x7f719c1f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f719be00700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f719c06b4a0 0x7f719c06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 -- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c13f700 msgr2=0x7f719c1f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c13f700 0x7f719c1f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 -- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f719c06b4a0 msgr2=0x7f719c06b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f719c06b4a0 0x7f719c06b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 -- 172.21.15.3:0/4210750548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c1f2d90 con 0x7f719c145bc0 2024-09-13T16:51:56.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7192a00700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f719c145bc0 0x7f719c1f2850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7184011700 tx=0x7f7184011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:56.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7193e00700 1 -- 172.21.15.3:0/4210750548 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718400c070 con 0x7f719c145bc0 2024-09-13T16:51:56.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7193e00700 1 -- 172.21.15.3:0/4210750548 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7184004220 con 0x7f719c145bc0 2024-09-13T16:51:56.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c143600 con 0x7f719c145bc0 2024-09-13T16:51:56.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f7193e00700 1 -- 172.21.15.3:0/4210750548 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184015730 con 0x7f719c145bc0 2024-09-13T16:51:56.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c143bc0 con 0x7f719c145bc0 2024-09-13T16:51:56.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.032+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c15de10 con 0x7f719c145bc0 2024-09-13T16:51:56.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.036+0000 7f7193e00700 1 -- 172.21.15.3:0/4210750548 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7184015890 con 0x7f719c145bc0 2024-09-13T16:51:56.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.036+0000 7f7193e00700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7178066730 0x7f7178068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.036+0000 7f7193e00700 1 -- 172.21.15.3:0/4210750548 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7184083e20 con 0x7f719c145bc0 2024-09-13T16:51:56.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.036+0000 7f719b400700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7178066730 0x7f7178068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.040+0000 7f719b400700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7178066730 0x7f7178068bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f719c0b28b0 tx=0x7f718c00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.044+0000 7f7193e00700 1 -- 172.21.15.3:0/4210750548 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7184051310 con 0x7f719c145bc0 2024-09-13T16:51:56.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.296+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f719c146330 con 0x7f719c145bc0 2024-09-13T16:51:56.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.296+0000 7f7193e00700 1 -- 172.21.15.3:0/4210750548 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f7184054930 con 0x7f719c145bc0 2024-09-13T16:51:56.303 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRm7PYhFhAAjGhG4+MGhybgDuUm80b3GA==", 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.304 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmwv/tKRAAkP5eDK1/mzVXdvg14IIyBQ==", 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmv2QBIRAAFCBAGuVsMDrEEbFHboF5fA==", 2024-09-13T16:51:56.305 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmqBN/NBAAcrl9JO1UMHIeJVdkAmMzBQ==", 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:51:56.307 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:51:56.307 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:51:56.307 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.307 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.307 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.307 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.307 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-13T16:51:56.307 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAfbeRmtMg4JBAAhpD6J8l0s10MToeuFpzBnQ==", 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-13T16:51:56.308 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAibeRmxu+sMxAAdKOLbj/zALwuUdqEcMGKOA==", 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAmbeRmneXPCBAA+5aq4baFkLLf9YgyT1Ej7A==", 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:51:56.309 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQApbeRmEJ5PGBAAm4ESpLC8pDcykDWH61UX+Q==", 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.310 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAtbeRmJ1v/NRAAVnN6OX537lXr9C2U5OwVcg==", 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAxbeRm1IErChAAlZJr0hgRM7hMjV9IKbFn5Q==", 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:51:56.311 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA0beRmjcW0FxAA8jqbdFW19Enk37M00fT5Mg==", 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA3beRmEURxJhAAbjA9Q82Jn0eyhkHLV3DnXg==", 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRm7UDIOhAAQ85WuAGFt3D40/vdHCx/SQ==", 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-13T16:51:56.313 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAcbeRm869WCxAALRQBL9frcnCHq6I4dwzKOA==", 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.314 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAcbeRmpROABRAA7JM6b7+quVMQ4WAlZ5Z0Kg==", 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAcbeRmYTniEBAALAPqutUd7WDZ6lUN3Zk4iQ==", 2024-09-13T16:51:56.315 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAabeRm2q6YKRAAmAaMZZqHOf6hMHtPpgmMPQ==", 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:51:56.316 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mds", 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA+beRm6j9ILBAAvYcjPSTo6kaM+Guz68eoCg==", 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mds" 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mgr", 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA+beRmRlZILBAACA7ua4JB8cL0X77kRQQk4w==", 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.317 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-osd", 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA+beRmYmdILBAAmbTTX2iQ27KcwEN0Lg27sQ==", 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-osd" 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd", 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA+beRm23dILBAAfXRNlqWLyKR7/mVEmPyvtw==", 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-13T16:51:56.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA+beRmoYhILBAA3jDQaMtHlb/lyVumpu2cIw==", 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rgw", 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA+beRmgZlILBAAS/kIAnBTuDx5T5petueP5A==", 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-13T16:51:56.319 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmEAD+DhAAwiQnsaTbFldMIXOfTHEdow==", 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmpP++CBAAYq8DHAVQrKZUzQMWAjwOQA==", 2024-09-13T16:51:56.320 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:51:56.321 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-13T16:51:56.321 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-13T16:51:56.321 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:51:56.321 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.321 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:51:56.321 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-13T16:51:56.321 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7178066730 msgr2=0x7f7178068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7178066730 0x7f7178068bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f719c0b28b0 tx=0x7f718c00a000 comp rx=0 tx=0).stop 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f719c145bc0 msgr2=0x7f719c1f2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f719c145bc0 0x7f719c1f2850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7184011700 tx=0x7f7184011ac0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 shutdown_connections 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7178066730 0x7f7178068bf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f719c06b4a0 0x7f719c06b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f719c145bc0 0x7f719c1f2850 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 --2- 172.21.15.3:0/4210750548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719c13f700 0x7f719c1f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 >> 172.21.15.3:0/4210750548 conn(0x7f719c0b2910 msgr2=0x7f719c1422d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 shutdown_connections 2024-09-13T16:51:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.300+0000 7f71a1200700 1 -- 172.21.15.3:0/4210750548 wait complete. 2024-09-13T16:51:56.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-13T16:51:56.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.496+0000 7f0903400700 1 Processor -- start 2024-09-13T16:51:56.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.496+0000 7f0903400700 1 -- start start 2024-09-13T16:51:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0903400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08fc070110 0x7f08fc0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0903400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc0cf5b0 con 0x7f08fc146a00 2024-09-13T16:51:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0903400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc0cd720 con 0x7f08fc146620 2024-09-13T16:51:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0903400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0b43b0 con 0x7f08fc070110 2024-09-13T16:51:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fbe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f08fc146a00 0x7f08fc06fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37986/0 (socket says 172.21.15.3:37986) 2024-09-13T16:51:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fbe00700 1 -- 172.21.15.3:0/368210157 learned_addr learned my addr 172.21.15.3:0/368210157 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0901400700 1 --2- 172.21.15.3:0/368210157 >> v2:172.21.15.3:3300/0 conn(0x7f08fc070110 0x7f08fc0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fb400700 1 -- 172.21.15.3:0/368210157 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72987614 0 0) 0x7f08fc0cf5b0 con 0x7f08fc146a00 2024-09-13T16:51:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fb400700 1 -- 172.21.15.3:0/368210157 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08e40035d0 con 0x7f08fc146a00 2024-09-13T16:51:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fb400700 1 -- 172.21.15.3:0/368210157 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640011019 0 0) 0x7f08fc0cd720 con 0x7f08fc146620 2024-09-13T16:51:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fb400700 1 -- 172.21.15.3:0/368210157 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08fc0cf5b0 con 0x7f08fc146620 2024-09-13T16:51:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0901400700 1 -- 172.21.15.3:0/368210157 >> v1:172.21.15.157:6789/0 conn(0x7f08fc146620 legacy=0x7f08fc0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0901400700 1 -- 172.21.15.3:0/368210157 >> v1:172.21.15.3:6789/0 conn(0x7f08fc146a00 legacy=0x7f08fc06fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0901400700 1 -- 172.21.15.3:0/368210157 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc0b4690 con 0x7f08fc070110 2024-09-13T16:51:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0901400700 1 --2- 172.21.15.3:0/368210157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc070110 0x7f08fc0b3dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009d90 tx=0x7f08f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452a60c82dc79f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fb400700 1 -- 172.21.15.3:0/368210157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f400e070 con 0x7f08fc070110 2024-09-13T16:51:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fb400700 1 -- 172.21.15.3:0/368210157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f400bb20 con 0x7f08fc070110 2024-09-13T16:51:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f08fb400700 1 -- 172.21.15.3:0/368210157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4012430 con 0x7f08fc070110 2024-09-13T16:51:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0903400700 1 -- 172.21.15.3:0/368210157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc070110 msgr2=0x7f08fc0b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.500+0000 7f0903400700 1 --2- 172.21.15.3:0/368210157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc070110 0x7f08fc0b3dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009d90 tx=0x7f08f400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.504+0000 7f0903400700 1 -- 172.21.15.3:0/368210157 shutdown_connections 2024-09-13T16:51:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.504+0000 7f0903400700 1 --2- 172.21.15.3:0/368210157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc070110 0x7f08fc0b3dd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.504+0000 7f0903400700 1 -- 172.21.15.3:0/368210157 >> 172.21.15.3:0/368210157 conn(0x7f08fc0b2ad0 msgr2=0x7f08fc0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.504+0000 7f0903400700 1 -- 172.21.15.3:0/368210157 shutdown_connections 2024-09-13T16:51:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.504+0000 7f0903400700 1 -- 172.21.15.3:0/368210157 wait complete. 2024-09-13T16:51:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.504+0000 7f0903400700 1 Processor -- start 2024-09-13T16:51:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.504+0000 7f0903400700 1 -- start start 2024-09-13T16:51:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0903400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc070110 0x7f08fc1f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0903400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08fc146620 0x7f08fc1eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0903400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc146a00 0x7f08fc13f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0903400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc13ff10 con 0x7f08fc146a00 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0903400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc1f3a90 con 0x7f08fc070110 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0903400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc13d680 con 0x7f08fc146620 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc070110 0x7f08fc1f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc070110 0x7f08fc1f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:47940/0 (socket says 172.21.15.3:47940) 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 -- 172.21.15.3:0/3874744967 learned_addr learned my addr 172.21.15.3:0/3874744967 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0901400700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc146a00 0x7f08fc13f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f08fbe00700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08fc146620 0x7f08fc1eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 -- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08fc146620 msgr2=0x7f08fc1eff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08fc146620 0x7f08fc1eff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 -- 172.21.15.3:0/3874744967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc146a00 msgr2=0x7f08fc13f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc146a00 0x7f08fc13f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 -- 172.21.15.3:0/3874744967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4009a70 con 0x7f08fc070110 2024-09-13T16:51:56.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0900a00700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc070110 0x7f08fc1f3550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08f000c730 tx=0x7f08f000ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:56.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f08f8c00700 1 -- 172.21.15.3:0/3874744967 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f00052d0 con 0x7f08fc070110 2024-09-13T16:51:56.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f08f8c00700 1 -- 172.21.15.3:0/3874744967 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f0005450 con 0x7f08fc070110 2024-09-13T16:51:56.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc13d960 con 0x7f08fc070110 2024-09-13T16:51:56.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f08f8c00700 1 -- 172.21.15.3:0/3874744967 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f00038e0 con 0x7f08fc070110 2024-09-13T16:51:56.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.508+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc13df20 con 0x7f08fc070110 2024-09-13T16:51:56.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.512+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc0b3dd0 con 0x7f08fc070110 2024-09-13T16:51:56.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.512+0000 7f08f8c00700 1 -- 172.21.15.3:0/3874744967 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08f0003a40 con 0x7f08fc070110 2024-09-13T16:51:56.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.512+0000 7f08f8c00700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08dc066730 0x7f08dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.516+0000 7f08f8c00700 1 -- 172.21.15.3:0/3874744967 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f08f0083500 con 0x7f08fc070110 2024-09-13T16:51:56.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.516+0000 7f08fbe00700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08dc066730 0x7f08dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.516+0000 7f08fbe00700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08dc066730 0x7f08dc068bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f08fc160550 tx=0x7f08e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.524+0000 7f08f8c00700 1 -- 172.21.15.3:0/3874744967 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f08f00509f0 con 0x7f08fc070110 2024-09-13T16:51:56.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.768+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f08fc13d3c0 con 0x7f08fc070110 2024-09-13T16:51:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.776+0000 7f08f8c00700 1 -- 172.21.15.3:0/3874744967 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f08f0054010 con 0x7f08fc070110 2024-09-13T16:51:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08dc066730 msgr2=0x7f08dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08dc066730 0x7f08dc068bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f08fc160550 tx=0x7f08e8009040 comp rx=0 tx=0).stop 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc070110 msgr2=0x7f08fc1f3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc070110 0x7f08fc1f3550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08f000c730 tx=0x7f08f000ca40 comp rx=0 tx=0).stop 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 shutdown_connections 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f08dc066730 0x7f08dc068bf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08fc146a00 0x7f08fc13f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08fc146620 0x7f08fc1eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 --2- 172.21.15.3:0/3874744967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc070110 0x7f08fc1f3550 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 >> 172.21.15.3:0/3874744967 conn(0x7f08fc0b2ad0 msgr2=0x7f08fc15b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 shutdown_connections 2024-09-13T16:51:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.780+0000 7f0903400700 1 -- 172.21.15.3:0/3874744967 wait complete. 2024-09-13T16:51:56.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-13T16:51:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.988+0000 7f4f24400700 1 Processor -- start 2024-09-13T16:51:56.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.988+0000 7f4f24400700 1 -- start start 2024-09-13T16:51:56.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.988+0000 7f4f24400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f1c15b660 0x7f4f1c0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.988+0000 7f4f24400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0714c0 0x7f4f1c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:56.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.988+0000 7f4f24400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0b3450 con 0x7f4f1c0b2f40 2024-09-13T16:51:56.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.988+0000 7f4f24400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b40d0 con 0x7f4f1c15b660 2024-09-13T16:51:56.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.988+0000 7f4f24400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b3730 con 0x7f4f1c0714c0 2024-09-13T16:51:56.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f21a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f1c15b660 0x7f4f1c0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f21000700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4f1c0b2f40 0x7f4f1c070db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:40030/0 (socket says 172.21.15.3:40030) 2024-09-13T16:51:56.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f21000700 1 -- 172.21.15.3:0/2723719905 learned_addr learned my addr 172.21.15.3:0/2723719905 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:56.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f22400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0714c0 0x7f4f1c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:56.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f1be00700 1 -- 172.21.15.3:0/2723719905 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802491658 0 0) 0x7f4f1c0b3450 con 0x7f4f1c0b2f40 2024-09-13T16:51:56.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f1be00700 1 -- 172.21.15.3:0/2723719905 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f040035d0 con 0x7f4f1c0b2f40 2024-09-13T16:51:56.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f22400700 1 -- 172.21.15.3:0/2723719905 >> v2:172.21.15.157:3300/0 conn(0x7f4f1c15b660 msgr2=0x7f4f1c0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:56.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f22400700 1 --2- 172.21.15.3:0/2723719905 >> v2:172.21.15.157:3300/0 conn(0x7f4f1c15b660 0x7f4f1c0b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:56.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f22400700 1 -- 172.21.15.3:0/2723719905 >> v1:172.21.15.157:6789/0 conn(0x7f4f1c0b2f40 legacy=0x7f4f1c070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f22400700 1 -- 172.21.15.3:0/2723719905 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0b3a10 con 0x7f4f1c0714c0 2024-09-13T16:51:56.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f22400700 1 --2- 172.21.15.3:0/2723719905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0714c0 0x7f4f1c06c540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009dc0 tx=0x7f4f1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6d849b6085b267 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:56.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f1be00700 1 -- 172.21.15.3:0/2723719905 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1400e070 con 0x7f4f1c0714c0 2024-09-13T16:51:56.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f1be00700 1 -- 172.21.15.3:0/2723719905 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1400bae0 con 0x7f4f1c0714c0 2024-09-13T16:51:56.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.992+0000 7f4f1be00700 1 -- 172.21.15.3:0/2723719905 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1400be90 con 0x7f4f1c0714c0 2024-09-13T16:51:56.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 -- 172.21.15.3:0/2723719905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0714c0 msgr2=0x7f4f1c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:56.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 --2- 172.21.15.3:0/2723719905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0714c0 0x7f4f1c06c540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009dc0 tx=0x7f4f1400b330 comp rx=0 tx=0).stop 2024-09-13T16:51:57.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 -- 172.21.15.3:0/2723719905 shutdown_connections 2024-09-13T16:51:57.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 --2- 172.21.15.3:0/2723719905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0714c0 0x7f4f1c06c540 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 --2- 172.21.15.3:0/2723719905 >> v2:172.21.15.157:3300/0 conn(0x7f4f1c15b660 0x7f4f1c0b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 -- 172.21.15.3:0/2723719905 >> 172.21.15.3:0/2723719905 conn(0x7f4f1c06a390 msgr2=0x7f4f1c06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:57.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 -- 172.21.15.3:0/2723719905 shutdown_connections 2024-09-13T16:51:57.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 -- 172.21.15.3:0/2723719905 wait complete. 2024-09-13T16:51:57.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 Processor -- start 2024-09-13T16:51:57.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:56.996+0000 7f4f24400700 1 -- start start 2024-09-13T16:51:57.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f24400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f1c0714c0 0x7f4f1c1f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f24400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0b2f40 0x7f4f1c1f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f24400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f1c15b660 0x7f4f1c1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f24400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1539c0 con 0x7f4f1c15b660 2024-09-13T16:51:57.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f24400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1f7e60 con 0x7f4f1c0714c0 2024-09-13T16:51:57.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f24400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1f7720 con 0x7f4f1c0b2f40 2024-09-13T16:51:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f1c15b660 0x7f4f1c1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f1c15b660 0x7f4f1c1f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43324/0 (socket says 172.21.15.3:43324) 2024-09-13T16:51:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 -- 172.21.15.3:0/1831628232 learned_addr learned my addr 172.21.15.3:0/1831628232 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f21000700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0b2f40 0x7f4f1c1f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f21a00700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f1c0714c0 0x7f4f1c1f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 -- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0b2f40 msgr2=0x7f4f1c1f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0b2f40 0x7f4f1c1f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 -- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f1c0714c0 msgr2=0x7f4f1c1f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f1c0714c0 0x7f4f1c1f9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 -- 172.21.15.3:0/1831628232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f14009a70 con 0x7f4f1c15b660 2024-09-13T16:51:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f22400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f1c15b660 0x7f4f1c1f71e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0b3870 tx=0x7f4f14004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f19600700 1 -- 172.21.15.3:0/1831628232 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140041c0 con 0x7f4f1c15b660 2024-09-13T16:51:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f19600700 1 -- 172.21.15.3:0/1831628232 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f14020070 con 0x7f4f1c15b660 2024-09-13T16:51:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f19600700 1 -- 172.21.15.3:0/1831628232 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14004560 con 0x7f4f1c15b660 2024-09-13T16:51:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.000+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c1536e0 con 0x7f4f1c15b660 2024-09-13T16:51:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.004+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c150dd0 con 0x7f4f1c15b660 2024-09-13T16:51:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.004+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c0b2a00 con 0x7f4f1c15b660 2024-09-13T16:51:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.004+0000 7f4f19600700 1 -- 172.21.15.3:0/1831628232 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f14022030 con 0x7f4f1c15b660 2024-09-13T16:51:57.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.008+0000 7f4f19600700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4efc066730 0x7f4efc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.008+0000 7f4f19600700 1 -- 172.21.15.3:0/1831628232 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4f14085800 con 0x7f4f1c15b660 2024-09-13T16:51:57.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.008+0000 7f4f21a00700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4efc066730 0x7f4efc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.008+0000 7f4f21a00700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4efc066730 0x7f4efc068bf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009b50 tx=0x7f4f10009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:57.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.016+0000 7f4f19600700 1 -- 172.21.15.3:0/1831628232 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4f14056800 con 0x7f4f1c15b660 2024-09-13T16:51:57.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.260+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4f1c06d360 con 0x7f4f1c15b660 2024-09-13T16:51:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.268+0000 7f4f19600700 1 -- 172.21.15.3:0/1831628232 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4f14056390 con 0x7f4f1c15b660 2024-09-13T16:51:57.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4efc066730 msgr2=0x7f4efc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4efc066730 0x7f4efc068bf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009b50 tx=0x7f4f10009240 comp rx=0 tx=0).stop 2024-09-13T16:51:57.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f1c15b660 msgr2=0x7f4f1c1f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f1c15b660 0x7f4f1c1f71e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0b3870 tx=0x7f4f14004af0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 shutdown_connections 2024-09-13T16:51:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4efc066730 0x7f4efc068bf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f1c15b660 0x7f4f1c1f71e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f1c0b2f40 0x7f4f1c1f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 --2- 172.21.15.3:0/1831628232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f1c0714c0 0x7f4f1c1f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 >> 172.21.15.3:0/1831628232 conn(0x7f4f1c06a390 msgr2=0x7f4f1c0cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 shutdown_connections 2024-09-13T16:51:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.272+0000 7f4f24400700 1 -- 172.21.15.3:0/1831628232 wait complete. 2024-09-13T16:51:57.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-13T16:51:57.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.468+0000 7f5df7e00700 1 Processor -- start 2024-09-13T16:51:57.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.468+0000 7f5df7e00700 1 -- start start 2024-09-13T16:51:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df7e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5df806fb50 0x7f5df8154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df7e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df8155000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df7e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df80b46e0 con 0x7f5df80b4b70 2024-09-13T16:51:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df7e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df815c0f0 con 0x7f5df806fb50 2024-09-13T16:51:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df7e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8155540 con 0x7f5df806ff30 2024-09-13T16:51:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df8155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df6a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5df806fb50 0x7f5df8154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df8155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:33636/0 (socket says 172.21.15.3:33636) 2024-09-13T16:51:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df6000700 1 -- 172.21.15.3:0/797368406 learned_addr learned my addr 172.21.15.3:0/797368406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.472+0000 7f5df7400700 1 --1- 172.21.15.3:0/797368406 >> v1:172.21.15.157:6789/0 conn(0x7f5df80b4b70 0x7f5df815b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:40032/0 (socket says 172.21.15.3:40032) 2024-09-13T16:51:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df6000700 1 -- 172.21.15.3:0/797368406 >> v2:172.21.15.157:3300/0 conn(0x7f5df806fb50 msgr2=0x7f5df8154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df6000700 1 --2- 172.21.15.3:0/797368406 >> v2:172.21.15.157:3300/0 conn(0x7f5df806fb50 0x7f5df8154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df6000700 1 -- 172.21.15.3:0/797368406 >> v1:172.21.15.157:6789/0 conn(0x7f5df80b4b70 legacy=0x7f5df815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df5600700 1 -- 172.21.15.3:0/797368406 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511327988 0 0) 0x7f5df80b46e0 con 0x7f5df80b4b70 2024-09-13T16:51:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df6000700 1 -- 172.21.15.3:0/797368406 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80b3db0 con 0x7f5df806ff30 2024-09-13T16:51:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df6a00700 1 --2- 172.21.15.3:0/797368406 >> v2:172.21.15.157:3300/0 conn(0x7f5df806fb50 0x7f5df8154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df6000700 1 --2- 172.21.15.3:0/797368406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df8155000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009910 tx=0x7f5de4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63ef45b05408f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df5600700 1 -- 172.21.15.3:0/797368406 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4011830 con 0x7f5df806ff30 2024-09-13T16:51:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df5600700 1 -- 172.21.15.3:0/797368406 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de4011990 con 0x7f5df806ff30 2024-09-13T16:51:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df5600700 1 -- 172.21.15.3:0/797368406 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4011d40 con 0x7f5df806ff30 2024-09-13T16:51:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df7e00700 1 -- 172.21.15.3:0/797368406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 msgr2=0x7f5df8155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df7e00700 1 --2- 172.21.15.3:0/797368406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df8155000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009910 tx=0x7f5de4009c20 comp rx=0 tx=0).stop 2024-09-13T16:51:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df7e00700 1 -- 172.21.15.3:0/797368406 shutdown_connections 2024-09-13T16:51:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df7e00700 1 --2- 172.21.15.3:0/797368406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df8155000 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df7e00700 1 --2- 172.21.15.3:0/797368406 >> v2:172.21.15.157:3300/0 conn(0x7f5df806fb50 0x7f5df8154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df7e00700 1 -- 172.21.15.3:0/797368406 >> 172.21.15.3:0/797368406 conn(0x7f5df80b2910 msgr2=0x7f5df80b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df7e00700 1 -- 172.21.15.3:0/797368406 shutdown_connections 2024-09-13T16:51:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.476+0000 7f5df7e00700 1 -- 172.21.15.3:0/797368406 wait complete. 2024-09-13T16:51:57.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.480+0000 7f5df7e00700 1 Processor -- start 2024-09-13T16:51:57.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.480+0000 7f5df7e00700 1 -- start start 2024-09-13T16:51:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df806fb50 0x7f5df80d1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5df80b4b70 0x7f5df814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df81f6990 con 0x7f5df80b4b70 2024-09-13T16:51:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df81f5d10 con 0x7f5df806fb50 2024-09-13T16:51:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df81f5ff0 con 0x7f5df806ff30 2024-09-13T16:51:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5df80b4b70 0x7f5df814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df806fb50 0x7f5df80d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5df80b4b70 0x7f5df814f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43338/0 (socket says 172.21.15.3:43338) 2024-09-13T16:51:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7400700 1 -- 172.21.15.3:0/122308010 learned_addr learned my addr 172.21.15.3:0/122308010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:57.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df6000700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df6a00700 1 -- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 msgr2=0x7f5df814f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df6a00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df814f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df6a00700 1 -- 172.21.15.3:0/122308010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5df80b4b70 msgr2=0x7f5df814f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df6a00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5df80b4b70 0x7f5df814f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df6a00700 1 -- 172.21.15.3:0/122308010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de80094b0 con 0x7f5df806fb50 2024-09-13T16:51:57.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7400700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5df80b4b70 0x7f5df814f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:51:57.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df6a00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df806fb50 0x7f5df80d1030 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5de800e9b0 tx=0x7f5de800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5deea00700 1 -- 172.21.15.3:0/122308010 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800c640 con 0x7f5df806fb50 2024-09-13T16:51:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5deea00700 1 -- 172.21.15.3:0/122308010 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de800c7a0 con 0x7f5df806fb50 2024-09-13T16:51:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5deea00700 1 -- 172.21.15.3:0/122308010 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800cb50 con 0x7f5df806fb50 2024-09-13T16:51:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.484+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de4009570 con 0x7f5df806fb50 2024-09-13T16:51:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.488+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df81f65b0 con 0x7f5df806fb50 2024-09-13T16:51:57.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.488+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df815b3a0 con 0x7f5df806fb50 2024-09-13T16:51:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.492+0000 7f5deea00700 1 -- 172.21.15.3:0/122308010 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5de800ccb0 con 0x7f5df806fb50 2024-09-13T16:51:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.492+0000 7f5deea00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5dd8066730 0x7f5dd8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.492+0000 7f5deea00700 1 -- 172.21.15.3:0/122308010 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5de80842a0 con 0x7f5df806fb50 2024-09-13T16:51:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.492+0000 7f5df6000700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5dd8066730 0x7f5dd8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.492+0000 7f5df6000700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5dd8066730 0x7f5dd8068bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5de4014040 tx=0x7f5de4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:57.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.500+0000 7f5deea00700 1 -- 172.21.15.3:0/122308010 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5de8051890 con 0x7f5df806fb50 2024-09-13T16:51:57.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.744+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f5df815ab70 con 0x7f5df806fb50 2024-09-13T16:51:57.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.756+0000 7f5deea00700 1 -- 172.21.15.3:0/122308010 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5de8054eb0 con 0x7f5df806fb50 2024-09-13T16:51:57.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5dd8066730 msgr2=0x7f5dd8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5dd8066730 0x7f5dd8068bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5de4014040 tx=0x7f5de4008030 comp rx=0 tx=0).stop 2024-09-13T16:51:57.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df806fb50 msgr2=0x7f5df80d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df806fb50 0x7f5df80d1030 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5de800e9b0 tx=0x7f5de800ecc0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 shutdown_connections 2024-09-13T16:51:57.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5dd8066730 0x7f5dd8068bf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5df80b4b70 0x7f5df814f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df806ff30 0x7f5df814f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 --2- 172.21.15.3:0/122308010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df806fb50 0x7f5df80d1030 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 >> 172.21.15.3:0/122308010 conn(0x7f5df80b2910 msgr2=0x7f5df80b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:57.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 shutdown_connections 2024-09-13T16:51:57.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.760+0000 7f5df7e00700 1 -- 172.21.15.3:0/122308010 wait complete. 2024-09-13T16:51:57.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-13T16:51:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.960+0000 7f57c3200700 1 Processor -- start 2024-09-13T16:51:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.960+0000 7f57c3200700 1 -- start start 2024-09-13T16:51:57.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c3200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57c406fb50 0x7f57c40cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c3200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 0x7f57c40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c3200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c40cf070 con 0x7f57c40b4390 2024-09-13T16:51:57.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c3200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c40b4be0 con 0x7f57c406fb50 2024-09-13T16:51:57.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c3200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40cf330 con 0x7f57c406ff30 2024-09-13T16:51:57.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c2800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f57c40b4390 0x7f57c415e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38000/0 (socket says 172.21.15.3:38000) 2024-09-13T16:51:57.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c2800700 1 -- 172.21.15.3:0/3323931431 learned_addr learned my addr 172.21.15.3:0/3323931431 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:57.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c1e00700 1 --2- 172.21.15.3:0/3323931431 >> v2:172.21.15.157:3300/0 conn(0x7f57c406fb50 0x7f57c40cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c1400700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 0x7f57c40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730849249 0 0) 0x7f57c40cf070 con 0x7f57c40b4390 2024-09-13T16:51:57.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a80035d0 con 0x7f57c40b4390 2024-09-13T16:51:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2359765362 0 0) 0x7f57a80035d0 con 0x7f57c40b4390 2024-09-13T16:51:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 msgr2=0x7f57c40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 0x7f57c40b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 >> v2:172.21.15.157:3300/0 conn(0x7f57c406fb50 msgr2=0x7f57c40cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 --2- 172.21.15.3:0/3323931431 >> v2:172.21.15.157:3300/0 conn(0x7f57c406fb50 0x7f57c40cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40cf610 con 0x7f57c40b4390 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f57bc0019f0 con 0x7f57c40b4390 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.964+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 >> v1:172.21.15.3:6789/0 conn(0x7f57c40b4390 legacy=0x7f57c415e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 0x7f57a8003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a8003fa0 0x7f57a8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8004db0 0x7f57a8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40b4be0 con 0x7f57a8003fa0 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40cf330 con 0x7f57a8004db0 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a800ba80 con 0x7f57c406ff30 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1400700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a8003fa0 0x7f57a8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1e00700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 0x7f57a8003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c2800700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8004db0 0x7f57a8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1400700 1 -- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 msgr2=0x7f57a8003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1400700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 0x7f57a8003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1400700 1 -- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8004db0 msgr2=0x7f57a8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1400700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8004db0 0x7f57a8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1400700 1 -- 172.21.15.3:0/3323931431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80035d0 con 0x7f57a8003fa0 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1e00700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ff30 0x7f57a8003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c1400700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a8003fa0 0x7f57a8004870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ba0 tx=0x7f57a80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d095486daf3a5e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000e6c0 con 0x7f57a8003fa0 2024-09-13T16:51:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b000e820 con 0x7f57a8003fa0 2024-09-13T16:51:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c0a00700 1 -- 172.21.15.3:0/3323931431 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000eb90 con 0x7f57a8003fa0 2024-09-13T16:51:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c3200700 1 -- 172.21.15.3:0/3323931431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a8003fa0 msgr2=0x7f57a8004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c3200700 1 --2- 172.21.15.3:0/3323931431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a8003fa0 0x7f57a8004870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ba0 tx=0x7f57a80058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c2800700 1 -- 172.21.15.3:0/3323931431 reap_dead start 2024-09-13T16:51:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c3200700 1 -- 172.21.15.3:0/3323931431 shutdown_connections 2024-09-13T16:51:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c3200700 1 -- 172.21.15.3:0/3323931431 >> 172.21.15.3:0/3323931431 conn(0x7f57c40b2910 msgr2=0x7f57c40b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c3200700 1 -- 172.21.15.3:0/3323931431 shutdown_connections 2024-09-13T16:51:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.968+0000 7f57c3200700 1 -- 172.21.15.3:0/3323931431 wait complete. 2024-09-13T16:51:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.972+0000 7f57c3200700 1 Processor -- start 2024-09-13T16:51:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.972+0000 7f57c3200700 1 -- start start 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c3200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8003fa0 0x7f57c4157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c3200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ed00 0x7f57c4142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c3200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406d2a0 0x7f57c41426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c3200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c406e450 con 0x7f57c406d2a0 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c3200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c406d840 con 0x7f57a8003fa0 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c3200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c406db20 con 0x7f57c406ed00 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406d2a0 0x7f57c41426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406d2a0 0x7f57c41426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43352/0 (socket says 172.21.15.3:43352) 2024-09-13T16:51:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 -- 172.21.15.3:0/2669422381 learned_addr learned my addr 172.21.15.3:0/2669422381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:57.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c1400700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ed00 0x7f57c4142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c1e00700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8003fa0 0x7f57c4157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 -- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ed00 msgr2=0x7f57c4142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ed00 0x7f57c4142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 -- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8003fa0 msgr2=0x7f57c4157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8003fa0 0x7f57c4157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 -- 172.21.15.3:0/2669422381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c406de00 con 0x7f57c406d2a0 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c2800700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406d2a0 0x7f57c41426d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f57bc000f80 tx=0x7f57bc001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57ba000700 1 -- 172.21.15.3:0/2669422381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00e810 con 0x7f57c406d2a0 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57ba000700 1 -- 172.21.15.3:0/2669422381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00e9b0 con 0x7f57c406d2a0 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57ba000700 1 -- 172.21.15.3:0/2669422381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00ed60 con 0x7f57c406d2a0 2024-09-13T16:51:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c406e0e0 con 0x7f57c406d2a0 2024-09-13T16:51:57.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.976+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c4157f70 con 0x7f57c406d2a0 2024-09-13T16:51:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.980+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c406e5c0 con 0x7f57c406d2a0 2024-09-13T16:51:57.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.980+0000 7f57ba000700 1 -- 172.21.15.3:0/2669422381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57bc01d460 con 0x7f57c406d2a0 2024-09-13T16:51:57.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.984+0000 7f57ba000700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f57a4066730 0x7f57a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:57.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.984+0000 7f57ba000700 1 -- 172.21.15.3:0/2669422381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f57bc084150 con 0x7f57c406d2a0 2024-09-13T16:51:57.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.984+0000 7f57c1e00700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f57a4066730 0x7f57a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:57.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.984+0000 7f57c1e00700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f57a4066730 0x7f57a4068bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f57c4157270 tx=0x7f57b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:57.992+0000 7f57ba000700 1 -- 172.21.15.3:0/2669422381 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f57bc051630 con 0x7f57c406d2a0 2024-09-13T16:51:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.244+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f57c4158160 con 0x7f57c406d2a0 2024-09-13T16:51:58.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.252+0000 7f57ba000700 1 -- 172.21.15.3:0/2669422381 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f57bc054c50 con 0x7f57c406d2a0 2024-09-13T16:51:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f57a4066730 msgr2=0x7f57a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f57a4066730 0x7f57a4068bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f57c4157270 tx=0x7f57b4008040 comp rx=0 tx=0).stop 2024-09-13T16:51:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406d2a0 msgr2=0x7f57c41426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406d2a0 0x7f57c41426d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f57bc000f80 tx=0x7f57bc001b00 comp rx=0 tx=0).stop 2024-09-13T16:51:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 shutdown_connections 2024-09-13T16:51:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f57a4066730 0x7f57a4068bf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406d2a0 0x7f57c41426d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57c406ed00 0x7f57c4142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 --2- 172.21.15.3:0/2669422381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a8003fa0 0x7f57c4157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 >> 172.21.15.3:0/2669422381 conn(0x7f57c40b2910 msgr2=0x7f57c4155970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 shutdown_connections 2024-09-13T16:51:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.256+0000 7f57c3200700 1 -- 172.21.15.3:0/2669422381 wait complete. 2024-09-13T16:51:58.283 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-13T16:51:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.460+0000 7fc8cba00700 1 Processor -- start 2024-09-13T16:51:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.460+0000 7fc8cba00700 1 -- start start 2024-09-13T16:51:58.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.460+0000 7fc8cba00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8c415b660 0x7fc8c40b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.460+0000 7fc8cba00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc8c40714c0 0x7fc8c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.460+0000 7fc8cba00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c40b3450 con 0x7fc8c40b2f40 2024-09-13T16:51:58.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.460+0000 7fc8cba00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c40b40d0 con 0x7fc8c40714c0 2024-09-13T16:51:58.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.460+0000 7fc8cba00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c40b3730 con 0x7fc8c415b660 2024-09-13T16:51:58.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc8c40714c0 0x7fc8c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc8c40714c0 0x7fc8c406c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43356/0 (socket says 172.21.15.3:43356) 2024-09-13T16:51:58.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9a00700 1 -- 172.21.15.3:0/3379261064 learned_addr learned my addr 172.21.15.3:0/3379261064 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:58.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9000700 1 --2- 172.21.15.3:0/3379261064 >> v2:172.21.15.157:3300/0 conn(0x7fc8c415b660 0x7fc8c40b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9a00700 1 -- 172.21.15.3:0/3379261064 >> v2:172.21.15.157:3300/0 conn(0x7fc8c415b660 msgr2=0x7fc8c40b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:58.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9a00700 1 --2- 172.21.15.3:0/3379261064 >> v2:172.21.15.157:3300/0 conn(0x7fc8c415b660 0x7fc8c40b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9a00700 1 -- 172.21.15.3:0/3379261064 >> v1:172.21.15.3:6789/0 conn(0x7fc8c40b2f40 legacy=0x7fc8c4070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9a00700 1 -- 172.21.15.3:0/3379261064 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c40b3a10 con 0x7fc8c40714c0 2024-09-13T16:51:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c9a00700 1 --2- 172.21.15.3:0/3379261064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40714c0 0x7fc8c406c540 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009e50 tx=0x7fc8bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c5bd35190173d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c3400700 1 -- 172.21.15.3:0/3379261064 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00e070 con 0x7fc8c40714c0 2024-09-13T16:51:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c3400700 1 -- 172.21.15.3:0/3379261064 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc00b8d0 con 0x7fc8c40714c0 2024-09-13T16:51:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8c3400700 1 -- 172.21.15.3:0/3379261064 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00bc80 con 0x7fc8c40714c0 2024-09-13T16:51:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8cba00700 1 -- 172.21.15.3:0/3379261064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40714c0 msgr2=0x7fc8c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8cba00700 1 --2- 172.21.15.3:0/3379261064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40714c0 0x7fc8c406c540 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009e50 tx=0x7fc8bc00b330 comp rx=0 tx=0).stop 2024-09-13T16:51:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8cba00700 1 -- 172.21.15.3:0/3379261064 shutdown_connections 2024-09-13T16:51:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8cba00700 1 --2- 172.21.15.3:0/3379261064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40714c0 0x7fc8c406c540 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8cba00700 1 --2- 172.21.15.3:0/3379261064 >> v2:172.21.15.157:3300/0 conn(0x7fc8c415b660 0x7fc8c40b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8cba00700 1 -- 172.21.15.3:0/3379261064 >> 172.21.15.3:0/3379261064 conn(0x7fc8c406a390 msgr2=0x7fc8c406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8cba00700 1 -- 172.21.15.3:0/3379261064 shutdown_connections 2024-09-13T16:51:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.464+0000 7fc8cba00700 1 -- 172.21.15.3:0/3379261064 wait complete. 2024-09-13T16:51:58.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.468+0000 7fc8cba00700 1 Processor -- start 2024-09-13T16:51:58.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.468+0000 7fc8cba00700 1 -- start start 2024-09-13T16:51:58.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.468+0000 7fc8cba00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c40714c0 0x7fc8c41f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.468+0000 7fc8cba00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40b2f40 0x7fc8c4153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.468+0000 7fc8cba00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c415b660 0x7fc8c4164c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.468+0000 7fc8cba00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c41659b0 con 0x7fc8c40b2f40 2024-09-13T16:51:58.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.468+0000 7fc8cba00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c41656d0 con 0x7fc8c40714c0 2024-09-13T16:51:58.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.468+0000 7fc8cba00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c41629e0 con 0x7fc8c415b660 2024-09-13T16:51:58.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40b2f40 0x7fc8c4153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40b2f40 0x7fc8c4153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43372/0 (socket says 172.21.15.3:43372) 2024-09-13T16:51:58.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 -- 172.21.15.3:0/4246835609 learned_addr learned my addr 172.21.15.3:0/4246835609 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:58.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c9000700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c40714c0 0x7fc8c41f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c9a00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c415b660 0x7fc8c4164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 -- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c415b660 msgr2=0x7fc8c4164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c415b660 0x7fc8c4164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 -- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c40714c0 msgr2=0x7fc8c41f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c40714c0 0x7fc8c41f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 -- 172.21.15.3:0/4246835609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc009b30 con 0x7fc8c40b2f40 2024-09-13T16:51:58.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c3e00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40b2f40 0x7fc8c4153ef0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c500 tx=0x7fc8b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:58.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c0c00700 1 -- 172.21.15.3:0/4246835609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00052d0 con 0x7fc8c40b2f40 2024-09-13T16:51:58.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c0c00700 1 -- 172.21.15.3:0/4246835609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0005430 con 0x7fc8c40b2f40 2024-09-13T16:51:58.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c4162cc0 con 0x7fc8c40b2f40 2024-09-13T16:51:58.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8c0c00700 1 -- 172.21.15.3:0/4246835609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00055a0 con 0x7fc8c40b2f40 2024-09-13T16:51:58.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c41f7fe0 con 0x7fc8c40b2f40 2024-09-13T16:51:58.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.472+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c4162840 con 0x7fc8c40b2f40 2024-09-13T16:51:58.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.476+0000 7fc8c0c00700 1 -- 172.21.15.3:0/4246835609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8b00036a0 con 0x7fc8c40b2f40 2024-09-13T16:51:58.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.476+0000 7fc8c0c00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8a4066730 0x7fc8a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.476+0000 7fc8c0c00700 1 -- 172.21.15.3:0/4246835609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc8b0083240 con 0x7fc8c40b2f40 2024-09-13T16:51:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.476+0000 7fc8c9000700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8a4066730 0x7fc8a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.480+0000 7fc8c9000700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8a4066730 0x7fc8a4068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009b50 tx=0x7fc8b8009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:58.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.484+0000 7fc8c0c00700 1 -- 172.21.15.3:0/4246835609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8c4162840 con 0x7fc8c40b2f40 2024-09-13T16:51:58.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:51:58.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:51:58.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.732+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc8c41631e0 con 0x7fc8c40b2f40 2024-09-13T16:51:58.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.740+0000 7fc8c0c00700 1 -- 172.21.15.3:0/4246835609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc8c41631e0 con 0x7fc8c40b2f40 2024-09-13T16:51:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8a4066730 msgr2=0x7fc8a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8a4066730 0x7fc8a4068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009b50 tx=0x7fc8b8009240 comp rx=0 tx=0).stop 2024-09-13T16:51:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40b2f40 msgr2=0x7fc8c4153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40b2f40 0x7fc8c4153ef0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c500 tx=0x7fc8b000c810 comp rx=0 tx=0).stop 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 shutdown_connections 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc8a4066730 0x7fc8a4068bf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c415b660 0x7fc8c4164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8c40b2f40 0x7fc8c4153ef0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 --2- 172.21.15.3:0/4246835609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c40714c0 0x7fc8c41f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 >> 172.21.15.3:0/4246835609 conn(0x7fc8c406a390 msgr2=0x7fc8c40ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 shutdown_connections 2024-09-13T16:51:58.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.744+0000 7fc8cba00700 1 -- 172.21.15.3:0/4246835609 wait complete. 2024-09-13T16:51:58.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-13T16:51:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.980+0000 7f3b8be00700 1 Processor -- start 2024-09-13T16:51:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8be00700 1 -- start start 2024-09-13T16:51:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b8c144790 0x7f3b8c06c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c06cb20 0x7f3b8c15b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8be00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8c0b4100 con 0x7f3b8c1443b0 2024-09-13T16:51:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8be00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8c06cfb0 con 0x7f3b8c144790 2024-09-13T16:51:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8be00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c0ce8e0 con 0x7f3b8c06cb20 2024-09-13T16:51:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8aa00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3b8c1443b0 0x7f3b8c0b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38018/0 (socket says 172.21.15.3:38018) 2024-09-13T16:51:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8aa00700 1 -- 172.21.15.3:0/624450030 learned_addr learned my addr 172.21.15.3:0/624450030 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8a000700 1 --2- 172.21.15.3:0/624450030 >> v2:172.21.15.3:3300/0 conn(0x7f3b8c144790 0x7f3b8c06c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8b400700 1 --2- 172.21.15.3:0/624450030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c06cb20 0x7f3b8c15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8a000700 1 -- 172.21.15.3:0/624450030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c06cb20 msgr2=0x7f3b8c15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:58.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8a000700 1 --2- 172.21.15.3:0/624450030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c06cb20 0x7f3b8c15b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b89600700 1 -- 172.21.15.3:0/624450030 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030645717 0 0) 0x7f3b8c0b4100 con 0x7f3b8c1443b0 2024-09-13T16:51:58.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8a000700 1 -- 172.21.15.3:0/624450030 >> v1:172.21.15.3:6789/0 conn(0x7f3b8c1443b0 legacy=0x7f3b8c0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.984+0000 7f3b8a000700 1 -- 172.21.15.3:0/624450030 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c0ccd30 con 0x7f3b8c144790 2024-09-13T16:51:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8a000700 1 --2- 172.21.15.3:0/624450030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c144790 0x7f3b8c06c5e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009850 tx=0x7f3b78009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd202a96212d5de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b89600700 1 -- 172.21.15.3:0/624450030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78011830 con 0x7f3b8c144790 2024-09-13T16:51:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b89600700 1 -- 172.21.15.3:0/624450030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b78011990 con 0x7f3b8c144790 2024-09-13T16:51:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b89600700 1 -- 172.21.15.3:0/624450030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78011d40 con 0x7f3b8c144790 2024-09-13T16:51:58.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8be00700 1 -- 172.21.15.3:0/624450030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c144790 msgr2=0x7f3b8c06c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:58.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8be00700 1 --2- 172.21.15.3:0/624450030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c144790 0x7f3b8c06c5e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009850 tx=0x7f3b78009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:58.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8be00700 1 -- 172.21.15.3:0/624450030 shutdown_connections 2024-09-13T16:51:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8be00700 1 --2- 172.21.15.3:0/624450030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c06cb20 0x7f3b8c15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8be00700 1 --2- 172.21.15.3:0/624450030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c144790 0x7f3b8c06c5e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8be00700 1 -- 172.21.15.3:0/624450030 >> 172.21.15.3:0/624450030 conn(0x7f3b8c0b0cf0 msgr2=0x7f3b8c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8be00700 1 -- 172.21.15.3:0/624450030 shutdown_connections 2024-09-13T16:51:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.988+0000 7f3b8be00700 1 -- 172.21.15.3:0/624450030 wait complete. 2024-09-13T16:51:58.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8be00700 1 Processor -- start 2024-09-13T16:51:58.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8be00700 1 -- start start 2024-09-13T16:51:58.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b8c06cb20 0x7f3b8c13a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c1443b0 0x7f3b8c140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8be00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c141100 0x7f3b8c1429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8be00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1f6500 con 0x7f3b8c1443b0 2024-09-13T16:51:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c141640 con 0x7f3b8c06cb20 2024-09-13T16:51:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8be00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1f5bd0 con 0x7f3b8c141100 2024-09-13T16:51:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8a000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c1443b0 0x7f3b8c140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8a000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c1443b0 0x7f3b8c140bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43394/0 (socket says 172.21.15.3:43394) 2024-09-13T16:51:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8a000700 1 -- 172.21.15.3:0/2866507460 learned_addr learned my addr 172.21.15.3:0/2866507460 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8aa00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b8c06cb20 0x7f3b8c13a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8b400700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c141100 0x7f3b8c1429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8a000700 1 -- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c141100 msgr2=0x7f3b8c1429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8a000700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c141100 0x7f3b8c1429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8a000700 1 -- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b8c06cb20 msgr2=0x7f3b8c13a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8a000700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b8c06cb20 0x7f3b8c13a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.992+0000 7f3b8a000700 1 -- 172.21.15.3:0/2866507460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780094b0 con 0x7f3b8c1443b0 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.996+0000 7f3b8a000700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c1443b0 0x7f3b8c140bc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3b78014040 tx=0x7f3b78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.996+0000 7f3b83400700 1 -- 172.21.15.3:0/2866507460 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78019ab0 con 0x7f3b8c1443b0 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.996+0000 7f3b83400700 1 -- 172.21.15.3:0/2866507460 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7801e070 con 0x7f3b8c1443b0 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.996+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c1f5eb0 con 0x7f3b8c1443b0 2024-09-13T16:51:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.996+0000 7f3b83400700 1 -- 172.21.15.3:0/2866507460 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78019e50 con 0x7f3b8c1443b0 2024-09-13T16:51:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.996+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c1f6990 con 0x7f3b8c1443b0 2024-09-13T16:51:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.996+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8c0cca20 con 0x7f3b8c1443b0 2024-09-13T16:51:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:58.996+0000 7f3b83400700 1 -- 172.21.15.3:0/2866507460 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3b7802d450 con 0x7f3b8c1443b0 2024-09-13T16:51:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.000+0000 7f3b83400700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b6c066730 0x7f3b6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.000+0000 7f3b83400700 1 -- 172.21.15.3:0/2866507460 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3b78084f60 con 0x7f3b8c1443b0 2024-09-13T16:51:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.000+0000 7f3b8aa00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b6c066730 0x7f3b6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.000+0000 7f3b8aa00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b6c066730 0x7f3b6c068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c000f60 tx=0x7f3b7c007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.004+0000 7f3b83400700 1 -- 172.21.15.3:0/2866507460 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3b78052570 con 0x7f3b8c1443b0 2024-09-13T16:51:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.248+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f3b8c1f6640 con 0x7f3b8c1443b0 2024-09-13T16:51:59.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.256+0000 7f3b83400700 1 -- 172.21.15.3:0/2866507460 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3b780150a0 con 0x7f3b8c1443b0 2024-09-13T16:51:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b6c066730 msgr2=0x7f3b6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b6c066730 0x7f3b6c068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c000f60 tx=0x7f3b7c007c00 comp rx=0 tx=0).stop 2024-09-13T16:51:59.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c1443b0 msgr2=0x7f3b8c140bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c1443b0 0x7f3b8c140bc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3b78014040 tx=0x7f3b78004770 comp rx=0 tx=0).stop 2024-09-13T16:51:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 shutdown_connections 2024-09-13T16:51:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3b6c066730 0x7f3b6c068bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b8c141100 0x7f3b8c1429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b8c1443b0 0x7f3b8c140bc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 --2- 172.21.15.3:0/2866507460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b8c06cb20 0x7f3b8c13a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 >> 172.21.15.3:0/2866507460 conn(0x7f3b8c0b0cf0 msgr2=0x7f3b8c0b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 shutdown_connections 2024-09-13T16:51:59.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.260+0000 7f3b8be00700 1 -- 172.21.15.3:0/2866507460 wait complete. 2024-09-13T16:51:59.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:51:59.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd562800700 1 Processor -- start 2024-09-13T16:51:59.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd562800700 1 -- start start 2024-09-13T16:51:59.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd562800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0b47a0 0x7fd55c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd562800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd55c15d1d0 0x7fd55c1460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd562800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55c15d660 con 0x7fd55c0cf820 2024-09-13T16:51:59.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd562800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c15d940 con 0x7fd55c15d1d0 2024-09-13T16:51:59.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd562800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c15dc20 con 0x7fd55c0b47a0 2024-09-13T16:51:59.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd561e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd55c0cf820 0x7fd55c1607e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38034/0 (socket says 172.21.15.3:38034) 2024-09-13T16:51:59.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd561e00700 1 -- 172.21.15.3:0/3698883354 learned_addr learned my addr 172.21.15.3:0/3698883354 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:59.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd560a00700 1 --2- 172.21.15.3:0/3698883354 >> v2:172.21.15.157:3300/0 conn(0x7fd55c15d1d0 0x7fd55c1460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd561400700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0b47a0 0x7fd55c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310006543 0 0) 0x7fd55c15d660 con 0x7fd55c0cf820 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5440035d0 con 0x7fd55c0cf820 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1920227099 0 0) 0x7fd5440035d0 con 0x7fd55c0cf820 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0b47a0 msgr2=0x7fd55c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0b47a0 0x7fd55c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 >> v2:172.21.15.157:3300/0 conn(0x7fd55c15d1d0 msgr2=0x7fd55c1460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 --2- 172.21.15.3:0/3698883354 >> v2:172.21.15.157:3300/0 conn(0x7fd55c15d1d0 0x7fd55c1460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c15df00 con 0x7fd55c0cf820 2024-09-13T16:51:59.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fd5540019f0 con 0x7fd55c0cf820 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 >> v1:172.21.15.3:6789/0 conn(0x7fd55c0cf820 legacy=0x7fd55c1607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0b47a0 0x7fd544003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd544004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544004db0 0x7fd544005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c15d940 con 0x7fd544003fa0 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c15dc20 con 0x7fd544004db0 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.472+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54400ba60 con 0x7fd55c0b47a0 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd560a00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd544004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd561400700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0b47a0 0x7fd544003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd561e00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544004db0 0x7fd544005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd560a00700 1 -- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0b47a0 msgr2=0x7fd544003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd560a00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0b47a0 0x7fd544003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd560a00700 1 -- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544004db0 msgr2=0x7fd544005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd560a00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544004db0 0x7fd544005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd560a00700 1 -- 172.21.15.3:0/3698883354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440035d0 con 0x7fd544003fa0 2024-09-13T16:51:59.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd560a00700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd544004870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd548009b70 tx=0x7fd5440058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828f82dbad74776a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:59.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800e6c0 con 0x7fd544003fa0 2024-09-13T16:51:59.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54800e820 con 0x7fd544003fa0 2024-09-13T16:51:59.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd55be00700 1 -- 172.21.15.3:0/3698883354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800eb90 con 0x7fd544003fa0 2024-09-13T16:51:59.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd562800700 1 -- 172.21.15.3:0/3698883354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 msgr2=0x7fd544004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd562800700 1 --2- 172.21.15.3:0/3698883354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd544004870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd548009b70 tx=0x7fd5440058a0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd561e00700 1 -- 172.21.15.3:0/3698883354 reap_dead start 2024-09-13T16:51:59.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd562800700 1 -- 172.21.15.3:0/3698883354 shutdown_connections 2024-09-13T16:51:59.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd562800700 1 -- 172.21.15.3:0/3698883354 >> 172.21.15.3:0/3698883354 conn(0x7fd55c071da0 msgr2=0x7fd55c0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:59.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd562800700 1 -- 172.21.15.3:0/3698883354 shutdown_connections 2024-09-13T16:51:59.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.476+0000 7fd562800700 1 -- 172.21.15.3:0/3698883354 wait complete. 2024-09-13T16:51:59.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.480+0000 7fd562800700 1 Processor -- start 2024-09-13T16:51:59.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.480+0000 7fd562800700 1 -- start start 2024-09-13T16:51:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.480+0000 7fd562800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd55c15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd562800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c1405a0 0x7fd55c0aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd562800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0ad8c0 0x7fd55c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd562800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0c4d80 con 0x7fd544003fa0 2024-09-13T16:51:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd562800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0ae280 con 0x7fd55c1405a0 2024-09-13T16:51:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd562800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c15cec0 con 0x7fd55c0ad8c0 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd55c15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd55c15c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43426/0 (socket says 172.21.15.3:43426) 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 -- 172.21.15.3:0/4139023938 learned_addr learned my addr 172.21.15.3:0/4139023938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd560a00700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c1405a0 0x7fd55c0aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561e00700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0ad8c0 0x7fd55c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 -- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0ad8c0 msgr2=0x7fd55c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0ad8c0 0x7fd55c0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 -- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c1405a0 msgr2=0x7fd55c0aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c1405a0 0x7fd55c0aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 -- 172.21.15.3:0/4139023938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0ab030 con 0x7fd544003fa0 2024-09-13T16:51:59.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd561400700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd55c15c980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd5500116d0 tx=0x7fd550011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:59.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd559600700 1 -- 172.21.15.3:0/4139023938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550015870 con 0x7fd544003fa0 2024-09-13T16:51:59.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd559600700 1 -- 172.21.15.3:0/4139023938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5500159d0 con 0x7fd544003fa0 2024-09-13T16:51:59.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd559600700 1 -- 172.21.15.3:0/4139023938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550015d60 con 0x7fd544003fa0 2024-09-13T16:51:59.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55c0ab610 con 0x7fd544003fa0 2024-09-13T16:51:59.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c0abbd0 con 0x7fd544003fa0 2024-09-13T16:51:59.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.484+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c0b4c10 con 0x7fd544003fa0 2024-09-13T16:51:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.488+0000 7fd559600700 1 -- 172.21.15.3:0/4139023938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd55001d400 con 0x7fd544003fa0 2024-09-13T16:51:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.488+0000 7fd559600700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd53c066730 0x7fd53c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.488+0000 7fd559600700 1 -- 172.21.15.3:0/4139023938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd550084600 con 0x7fd544003fa0 2024-09-13T16:51:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.488+0000 7fd560a00700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd53c066730 0x7fd53c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.488+0000 7fd560a00700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd53c066730 0x7fd53c068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd5480090b0 tx=0x7fd548008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.492+0000 7fd559600700 1 -- 172.21.15.3:0/4139023938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd550051b70 con 0x7fd544003fa0 2024-09-13T16:51:59.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.684+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd55c0ced50 con 0x7fd544003fa0 2024-09-13T16:51:59.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.688+0000 7fd559600700 1 -- 172.21.15.3:0/4139023938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13014 (secure 0 0 0) 0x7fd550055190 con 0x7fd544003fa0 2024-09-13T16:51:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:51:59.693 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:51:55.801852+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd53c066730 msgr2=0x7fd53c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd53c066730 0x7fd53c068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd5480090b0 tx=0x7fd548008030 comp rx=0 tx=0).stop 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 msgr2=0x7fd55c15c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd55c15c980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd5500116d0 tx=0x7fd550011a90 comp rx=0 tx=0).stop 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 shutdown_connections 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd53c066730 0x7fd53c068bf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd55c0ad8c0 0x7fd55c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c1405a0 0x7fd55c0aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 --2- 172.21.15.3:0/4139023938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd544003fa0 0x7fd55c15c980 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 >> 172.21.15.3:0/4139023938 conn(0x7fd55c071da0 msgr2=0x7fd55c0b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 shutdown_connections 2024-09-13T16:51:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.692+0000 7fd562800700 1 -- 172.21.15.3:0/4139023938 wait complete. 2024-09-13T16:51:59.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:51:59.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:51:59.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.896+0000 7fcac9200700 1 Processor -- start 2024-09-13T16:51:59.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac9200700 1 -- start start 2024-09-13T16:51:59.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac9200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcac415b1c0 0x7fcac40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac9200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac40cc830 0x7fcac40ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac9200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac40cd1c0 con 0x7fcac4158740 2024-09-13T16:51:59.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac9200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac40b3960 con 0x7fcac415b1c0 2024-09-13T16:51:59.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac9200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40b1f30 con 0x7fcac40cc830 2024-09-13T16:51:59.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac1400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcac415b1c0 0x7fcac40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac1e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcac4158740 0x7fcac40b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:40046/0 (socket says 172.21.15.3:40046) 2024-09-13T16:51:59.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac1e00700 1 -- 172.21.15.3:0/79812759 learned_addr learned my addr 172.21.15.3:0/79812759 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:51:59.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac1400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcac415b1c0 0x7fcac40ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43442/0 (socket says 172.21.15.3:43442) 2024-09-13T16:51:59.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac2800700 1 --2- 172.21.15.3:0/79812759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac40cc830 0x7fcac40ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac1400700 1 -- 172.21.15.3:0/79812759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac40cc830 msgr2=0x7fcac40ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:59.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac1400700 1 --2- 172.21.15.3:0/79812759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac40cc830 0x7fcac40ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac1400700 1 -- 172.21.15.3:0/79812759 >> v1:172.21.15.157:6789/0 conn(0x7fcac4158740 legacy=0x7fcac40b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac1400700 1 -- 172.21.15.3:0/79812759 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac415b710 con 0x7fcac415b1c0 2024-09-13T16:51:59.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.900+0000 7fcac0a00700 1 -- 172.21.15.3:0/79812759 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655872434 0 0) 0x7fcab4001110 con 0x7fcac4158740 2024-09-13T16:51:59.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac1400700 1 --2- 172.21.15.3:0/79812759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac415b1c0 0x7fcac40ce6e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009850 tx=0x7fcab0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bcde70566c9c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:59.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac0a00700 1 -- 172.21.15.3:0/79812759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0011830 con 0x7fcac415b1c0 2024-09-13T16:51:59.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac0a00700 1 -- 172.21.15.3:0/79812759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0011990 con 0x7fcac415b1c0 2024-09-13T16:51:59.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac0a00700 1 -- 172.21.15.3:0/79812759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0011d40 con 0x7fcac415b1c0 2024-09-13T16:51:59.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac9200700 1 -- 172.21.15.3:0/79812759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac415b1c0 msgr2=0x7fcac40ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac9200700 1 --2- 172.21.15.3:0/79812759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac415b1c0 0x7fcac40ce6e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009850 tx=0x7fcab0009b60 comp rx=0 tx=0).stop 2024-09-13T16:51:59.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac9200700 1 -- 172.21.15.3:0/79812759 shutdown_connections 2024-09-13T16:51:59.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac9200700 1 --2- 172.21.15.3:0/79812759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac40cc830 0x7fcac40ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac9200700 1 --2- 172.21.15.3:0/79812759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac415b1c0 0x7fcac40ce6e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac9200700 1 -- 172.21.15.3:0/79812759 >> 172.21.15.3:0/79812759 conn(0x7fcac406b280 msgr2=0x7fcac406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:59.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac9200700 1 -- 172.21.15.3:0/79812759 shutdown_connections 2024-09-13T16:51:59.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.904+0000 7fcac9200700 1 -- 172.21.15.3:0/79812759 wait complete. 2024-09-13T16:51:59.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac9200700 1 Processor -- start 2024-09-13T16:51:59.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac9200700 1 -- start start 2024-09-13T16:51:59.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac9200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40cc830 0x7fcac40adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac9200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac4158740 0x7fcac4156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac415b1c0 0x7fcac4157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac9200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4155d00 con 0x7fcac40cc830 2024-09-13T16:51:59.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac9200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4154ea0 con 0x7fcac415b1c0 2024-09-13T16:51:59.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac9200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4155180 con 0x7fcac4158740 2024-09-13T16:51:59.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40cc830 0x7fcac40adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40cc830 0x7fcac40adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43444/0 (socket says 172.21.15.3:43444) 2024-09-13T16:51:59.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 -- 172.21.15.3:0/3557522168 learned_addr learned my addr 172.21.15.3:0/3557522168 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:51:59.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac2800700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac415b1c0 0x7fcac4157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1400700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac4158740 0x7fcac4156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 -- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac4158740 msgr2=0x7fcac4156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac4158740 0x7fcac4156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 -- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac415b1c0 msgr2=0x7fcac4157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:59.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac415b1c0 0x7fcac4157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:59.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 -- 172.21.15.3:0/3557522168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00094b0 con 0x7fcac40cc830 2024-09-13T16:51:59.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.908+0000 7fcac1e00700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40cc830 0x7fcac40adb70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c4b0 tx=0x7fcab400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:59.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.912+0000 7fcaba000700 1 -- 172.21.15.3:0/3557522168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4005d90 con 0x7fcac40cc830 2024-09-13T16:51:59.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.912+0000 7fcaba000700 1 -- 172.21.15.3:0/3557522168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab40070d0 con 0x7fcac40cc830 2024-09-13T16:51:59.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.912+0000 7fcaba000700 1 -- 172.21.15.3:0/3557522168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4007460 con 0x7fcac40cc830 2024-09-13T16:51:59.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.912+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac4155460 con 0x7fcac40cc830 2024-09-13T16:51:59.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.912+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac4155a20 con 0x7fcac40cc830 2024-09-13T16:51:59.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.912+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac406bd70 con 0x7fcac40cc830 2024-09-13T16:51:59.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.916+0000 7fcaba000700 1 -- 172.21.15.3:0/3557522168 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcab40075c0 con 0x7fcac40cc830 2024-09-13T16:51:59.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.916+0000 7fcaba000700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcaa4066730 0x7fcaa4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:59.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.916+0000 7fcaba000700 1 -- 172.21.15.3:0/3557522168 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fcab4083250 con 0x7fcac40cc830 2024-09-13T16:51:59.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.916+0000 7fcac1400700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcaa4066730 0x7fcaa4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:59.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.916+0000 7fcac1400700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcaa4066730 0x7fcaa4068bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcab00090b0 tx=0x7fcab0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:51:59.924+0000 7fcaba000700 1 -- 172.21.15.3:0/3557522168 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcab4050860 con 0x7fcac40cc830 2024-09-13T16:52:00.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.104+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcac40cd170 con 0x7fcac40cc830 2024-09-13T16:52:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.108+0000 7fcaba000700 1 -- 172.21.15.3:0/3557522168 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcab4053e80 con 0x7fcac40cc830 2024-09-13T16:52:00.112 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:52:00.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcaa4066730 msgr2=0x7fcaa4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcaa4066730 0x7fcaa4068bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcab00090b0 tx=0x7fcab0008030 comp rx=0 tx=0).stop 2024-09-13T16:52:00.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40cc830 msgr2=0x7fcac40adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40cc830 0x7fcac40adb70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c4b0 tx=0x7fcab400c870 comp rx=0 tx=0).stop 2024-09-13T16:52:00.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 shutdown_connections 2024-09-13T16:52:00.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcaa4066730 0x7fcaa4068bf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac415b1c0 0x7fcac4157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac4158740 0x7fcac4156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 --2- 172.21.15.3:0/3557522168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40cc830 0x7fcac40adb70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 >> 172.21.15.3:0/3557522168 conn(0x7fcac406b280 msgr2=0x7fcac4158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 shutdown_connections 2024-09-13T16:52:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.112+0000 7fcac9200700 1 -- 172.21.15.3:0/3557522168 wait complete. 2024-09-13T16:52:00.138 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T16:52:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.304+0000 7feb26400700 1 Processor -- start 2024-09-13T16:52:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.304+0000 7feb26400700 1 -- start start 2024-09-13T16:52:00.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.304+0000 7feb26400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb2015cfa0 0x7feb200ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.304+0000 7feb26400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb200ae1e0 0x7feb200cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.304+0000 7feb26400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2006a390 con 0x7feb2015cbc0 2024-09-13T16:52:00.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.304+0000 7feb26400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2006a670 con 0x7feb2015cfa0 2024-09-13T16:52:00.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.304+0000 7feb26400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2006a950 con 0x7feb200ae1e0 2024-09-13T16:52:00.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb17e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb2015cfa0 0x7feb200ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb25000700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7feb2015cbc0 0x7feb200adad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:40056/0 (socket says 172.21.15.3:40056) 2024-09-13T16:52:00.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb25000700 1 -- 172.21.15.3:0/542350726 learned_addr learned my addr 172.21.15.3:0/542350726 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:00.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb25a00700 1 --2- 172.21.15.3:0/542350726 >> v2:172.21.15.157:3300/0 conn(0x7feb200ae1e0 0x7feb200cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb1fe00700 1 -- 172.21.15.3:0/542350726 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663467446 0 0) 0x7feb2006a390 con 0x7feb2015cbc0 2024-09-13T16:52:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb1fe00700 1 -- 172.21.15.3:0/542350726 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb000035d0 con 0x7feb2015cbc0 2024-09-13T16:52:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb17e00700 1 -- 172.21.15.3:0/542350726 >> v2:172.21.15.157:3300/0 conn(0x7feb200ae1e0 msgr2=0x7feb200cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb17e00700 1 --2- 172.21.15.3:0/542350726 >> v2:172.21.15.157:3300/0 conn(0x7feb200ae1e0 0x7feb200cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb17e00700 1 -- 172.21.15.3:0/542350726 >> v1:172.21.15.157:6789/0 conn(0x7feb2015cbc0 legacy=0x7feb200adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb17e00700 1 -- 172.21.15.3:0/542350726 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2006ac30 con 0x7feb2015cfa0 2024-09-13T16:52:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb17e00700 1 --2- 172.21.15.3:0/542350726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 0x7feb200ccb60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feb2006e0f0 tx=0x7feb08009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06c37497f95ed9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb1fe00700 1 -- 172.21.15.3:0/542350726 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08011830 con 0x7feb2015cfa0 2024-09-13T16:52:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb1fe00700 1 -- 172.21.15.3:0/542350726 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb08011990 con 0x7feb2015cfa0 2024-09-13T16:52:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb1fe00700 1 -- 172.21.15.3:0/542350726 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08011d20 con 0x7feb2015cfa0 2024-09-13T16:52:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb26400700 1 -- 172.21.15.3:0/542350726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 msgr2=0x7feb200ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb26400700 1 --2- 172.21.15.3:0/542350726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 0x7feb200ccb60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feb2006e0f0 tx=0x7feb08009b30 comp rx=0 tx=0).stop 2024-09-13T16:52:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb26400700 1 -- 172.21.15.3:0/542350726 shutdown_connections 2024-09-13T16:52:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb26400700 1 --2- 172.21.15.3:0/542350726 >> v2:172.21.15.157:3300/0 conn(0x7feb200ae1e0 0x7feb200cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb26400700 1 --2- 172.21.15.3:0/542350726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 0x7feb200ccb60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.308+0000 7feb26400700 1 -- 172.21.15.3:0/542350726 >> 172.21.15.3:0/542350726 conn(0x7feb200b0cf0 msgr2=0x7feb200b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 -- 172.21.15.3:0/542350726 shutdown_connections 2024-09-13T16:52:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 -- 172.21.15.3:0/542350726 wait complete. 2024-09-13T16:52:00.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 Processor -- start 2024-09-13T16:52:00.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 -- start start 2024-09-13T16:52:00.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200ae1e0 0x7feb201f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb2015cbc0 0x7feb2015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 0x7feb2014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb201503a0 con 0x7feb2015cfa0 2024-09-13T16:52:00.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015ece0 con 0x7feb200ae1e0 2024-09-13T16:52:00.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.312+0000 7feb26400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015e730 con 0x7feb2015cbc0 2024-09-13T16:52:00.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200ae1e0 0x7feb201f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 0x7feb2014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200ae1e0 0x7feb201f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:48124/0 (socket says 172.21.15.3:48124) 2024-09-13T16:52:00.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25000700 1 -- 172.21.15.3:0/720841878 learned_addr learned my addr 172.21.15.3:0/720841878 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:00.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb17e00700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb2015cbc0 0x7feb2015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25a00700 1 -- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb2015cbc0 msgr2=0x7feb2015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25a00700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb2015cbc0 0x7feb2015e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25a00700 1 -- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200ae1e0 msgr2=0x7feb201f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25a00700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200ae1e0 0x7feb201f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25a00700 1 -- 172.21.15.3:0/720841878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080094b0 con 0x7feb2015cfa0 2024-09-13T16:52:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb25a00700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 0x7feb2014d560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feb1800efc0 tx=0x7feb1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb1d600700 1 -- 172.21.15.3:0/720841878 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1800cc90 con 0x7feb2015cfa0 2024-09-13T16:52:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb1d600700 1 -- 172.21.15.3:0/720841878 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1800f040 con 0x7feb2015cfa0 2024-09-13T16:52:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2015ea00 con 0x7feb2015cfa0 2024-09-13T16:52:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb1d600700 1 -- 172.21.15.3:0/720841878 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18009e60 con 0x7feb2015cfa0 2024-09-13T16:52:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2014ffc0 con 0x7feb2015cfa0 2024-09-13T16:52:00.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.316+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb201504e0 con 0x7feb2015cfa0 2024-09-13T16:52:00.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.320+0000 7feb1d600700 1 -- 172.21.15.3:0/720841878 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feb18014480 con 0x7feb2015cfa0 2024-09-13T16:52:00.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.320+0000 7feb1d600700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7feafc066730 0x7feafc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.320+0000 7feb1d600700 1 -- 172.21.15.3:0/720841878 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7feb18084750 con 0x7feb2015cfa0 2024-09-13T16:52:00.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.320+0000 7feb25000700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7feafc066730 0x7feafc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.324+0000 7feb25000700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7feafc066730 0x7feafc068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feb100011e0 tx=0x7feb10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:00.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.324+0000 7feb1d600700 1 -- 172.21.15.3:0/720841878 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feb201504e0 con 0x7feb2015cfa0 2024-09-13T16:52:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.520+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7feb201504e0 con 0x7feb2015cfa0 2024-09-13T16:52:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.520+0000 7feb1d600700 1 -- 172.21.15.3:0/720841878 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7feb201504e0 con 0x7feb2015cfa0 2024-09-13T16:52:00.525 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:00.525 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-13T16:52:00.525 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:00.525 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-13T16:52:00.525 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-13T16:52:00.525 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:00.525 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-13T16:52:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7feafc066730 msgr2=0x7feafc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7feafc066730 0x7feafc068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feb100011e0 tx=0x7feb10009040 comp rx=0 tx=0).stop 2024-09-13T16:52:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 msgr2=0x7feb2014d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 0x7feb2014d560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feb1800efc0 tx=0x7feb1800c3a0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 shutdown_connections 2024-09-13T16:52:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7feafc066730 0x7feafc068bf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2015cfa0 0x7feb2014d560 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb2015cbc0 0x7feb2015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 --2- 172.21.15.3:0/720841878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200ae1e0 0x7feb201f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 >> 172.21.15.3:0/720841878 conn(0x7feb200b0cf0 msgr2=0x7feb200ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 shutdown_connections 2024-09-13T16:52:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.524+0000 7feb26400700 1 -- 172.21.15.3:0/720841878 wait complete. 2024-09-13T16:52:00.551 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T16:52:00.551 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T16:52:00.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.724+0000 7f367f200700 1 Processor -- start 2024-09-13T16:52:00.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.724+0000 7f367f200700 1 -- start start 2024-09-13T16:52:00.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367f200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36800b1400 0x7f36801653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367f200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368006cf40 con 0x7f36800b1020 2024-09-13T16:52:00.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367f200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368006d1e0 con 0x7f36800cec90 2024-09-13T16:52:00.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367f200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368006ba60 con 0x7f36800b1400 2024-09-13T16:52:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367de00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f36800b1020 0x7f368006b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38040/0 (socket says 172.21.15.3:38040) 2024-09-13T16:52:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367de00700 1 -- 172.21.15.3:0/1045763933 learned_addr learned my addr 172.21.15.3:0/1045763933 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367e800700 1 --2- 172.21.15.3:0/1045763933 >> v2:172.21.15.157:3300/0 conn(0x7f36800b1400 0x7f36801653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367ca00700 1 -- 172.21.15.3:0/1045763933 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930364535 0 0) 0x7f368006cf40 con 0x7f36800b1020 2024-09-13T16:52:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367ca00700 1 -- 172.21.15.3:0/1045763933 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36640035d0 con 0x7f36800b1020 2024-09-13T16:52:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367ca00700 1 -- 172.21.15.3:0/1045763933 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878848786 0 0) 0x7f368006d1e0 con 0x7f36800cec90 2024-09-13T16:52:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367ca00700 1 -- 172.21.15.3:0/1045763933 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f368006cf40 con 0x7f36800cec90 2024-09-13T16:52:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367e800700 1 -- 172.21.15.3:0/1045763933 >> v1:172.21.15.157:6789/0 conn(0x7f36800cec90 legacy=0x7f3680164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367e800700 1 -- 172.21.15.3:0/1045763933 >> v1:172.21.15.3:6789/0 conn(0x7f36800b1020 legacy=0x7f368006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367e800700 1 -- 172.21.15.3:0/1045763933 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800cf190 con 0x7f36800b1400 2024-09-13T16:52:00.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367e800700 1 --2- 172.21.15.3:0/1045763933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1400 0x7f36801653e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3678009dc0 tx=0x7f367800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2cbcf1f5a1e43c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:00.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367ca00700 1 -- 172.21.15.3:0/1045763933 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800e070 con 0x7f36800b1400 2024-09-13T16:52:00.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367ca00700 1 -- 172.21.15.3:0/1045763933 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367800bae0 con 0x7f36800b1400 2024-09-13T16:52:00.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.728+0000 7f367ca00700 1 -- 172.21.15.3:0/1045763933 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800be90 con 0x7f36800b1400 2024-09-13T16:52:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.732+0000 7f367f200700 1 -- 172.21.15.3:0/1045763933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1400 msgr2=0x7f36801653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.732+0000 7f367f200700 1 --2- 172.21.15.3:0/1045763933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1400 0x7f36801653e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3678009dc0 tx=0x7f367800b330 comp rx=0 tx=0).stop 2024-09-13T16:52:00.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.732+0000 7f367f200700 1 -- 172.21.15.3:0/1045763933 shutdown_connections 2024-09-13T16:52:00.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.732+0000 7f367f200700 1 --2- 172.21.15.3:0/1045763933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1400 0x7f36801653e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.732+0000 7f367f200700 1 -- 172.21.15.3:0/1045763933 >> 172.21.15.3:0/1045763933 conn(0x7f36800af960 msgr2=0x7f36800b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:00.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.732+0000 7f367f200700 1 -- 172.21.15.3:0/1045763933 shutdown_connections 2024-09-13T16:52:00.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.732+0000 7f367f200700 1 -- 172.21.15.3:0/1045763933 wait complete. 2024-09-13T16:52:00.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367f200700 1 Processor -- start 2024-09-13T16:52:00.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367f200700 1 -- start start 2024-09-13T16:52:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1020 0x7f368015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367f200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36800b1400 0x7f3680162d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367f200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36800cec90 0x7f3680163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367f200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800af580 con 0x7f36800b1400 2024-09-13T16:52:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3680163f40 con 0x7f36800b1020 2024-09-13T16:52:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367f200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800ae590 con 0x7f36800cec90 2024-09-13T16:52:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36800b1400 0x7f3680162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36800b1400 0x7f3680162d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43472/0 (socket says 172.21.15.3:43472) 2024-09-13T16:52:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1020 0x7f368015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367d400700 1 -- 172.21.15.3:0/49820553 learned_addr learned my addr 172.21.15.3:0/49820553 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367e800700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36800cec90 0x7f3680163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367d400700 1 -- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36800cec90 msgr2=0x7f3680163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367d400700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36800cec90 0x7f3680163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367d400700 1 -- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1020 msgr2=0x7f368015c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367d400700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1020 0x7f368015c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.736+0000 7f367d400700 1 -- 172.21.15.3:0/49820553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678009a70 con 0x7f36800b1400 2024-09-13T16:52:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.740+0000 7f367d400700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36800b1400 0x7f3680162d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c500 tx=0x7f366c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.740+0000 7f3676000700 1 -- 172.21.15.3:0/49820553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c0052d0 con 0x7f36800b1400 2024-09-13T16:52:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.740+0000 7f3676000700 1 -- 172.21.15.3:0/49820553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c005450 con 0x7f36800b1400 2024-09-13T16:52:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.740+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36800ac6d0 con 0x7f36800b1400 2024-09-13T16:52:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.740+0000 7f3676000700 1 -- 172.21.15.3:0/49820553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c0038e0 con 0x7f36800b1400 2024-09-13T16:52:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.740+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368015c7d0 con 0x7f36800b1400 2024-09-13T16:52:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.740+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368006e740 con 0x7f36800b1400 2024-09-13T16:52:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.744+0000 7f3676000700 1 -- 172.21.15.3:0/49820553 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f366c003a40 con 0x7f36800b1400 2024-09-13T16:52:00.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.744+0000 7f3676000700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3660066730 0x7f3660068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:00.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.744+0000 7f3676000700 1 -- 172.21.15.3:0/49820553 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f366c082f70 con 0x7f36800b1400 2024-09-13T16:52:00.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.744+0000 7f367de00700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3660066730 0x7f3660068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.748+0000 7f367de00700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3660066730 0x7f3660068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3670007fd0 tx=0x7f3670007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.752+0000 7f3676000700 1 -- 172.21.15.3:0/49820553 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f366c050450 con 0x7f36800b1400 2024-09-13T16:52:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:00.936+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f36800ac6d0 con 0x7f36800b1400 2024-09-13T16:52:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.552+0000 7f3676000700 1 -- 172.21.15.3:0/49820553 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f36800ac6d0 con 0x7f36800b1400 2024-09-13T16:52:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3660066730 msgr2=0x7f3660068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3660066730 0x7f3660068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3670007fd0 tx=0x7f3670007de0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36800b1400 msgr2=0x7f3680162d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36800b1400 0x7f3680162d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c500 tx=0x7f366c00c810 comp rx=0 tx=0).stop 2024-09-13T16:52:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 shutdown_connections 2024-09-13T16:52:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3660066730 0x7f3660068bf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36800cec90 0x7f3680163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36800b1400 0x7f3680162d40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 --2- 172.21.15.3:0/49820553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800b1020 0x7f368015c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 >> 172.21.15.3:0/49820553 conn(0x7f36800af960 msgr2=0x7f36800b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 shutdown_connections 2024-09-13T16:52:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.560+0000 7f367f200700 1 -- 172.21.15.3:0/49820553 wait complete. 2024-09-13T16:52:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-13T16:52:01.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T16:52:01.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcd0a00700 1 Processor -- start 2024-09-13T16:52:01.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcd0a00700 1 -- start start 2024-09-13T16:52:01.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcd0a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdccc1444f0 0x7fdccc0d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcd0a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0d0680 0x7fdccc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcd0a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0b5a30 con 0x7fdccc0b5570 2024-09-13T16:52:01.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcd0a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdccc0b5d10 con 0x7fdccc1444f0 2024-09-13T16:52:01.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcd0a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0cd5b0 con 0x7fdccc0d0680 2024-09-13T16:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdccbe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdccc0b5570 0x7fdccc15b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38054/0 (socket says 172.21.15.3:38054) 2024-09-13T16:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdccbe00700 1 -- 172.21.15.3:0/4188313504 learned_addr learned my addr 172.21.15.3:0/4188313504 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdccaa00700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0d0680 0x7fdccc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdccb400700 1 --2- 172.21.15.3:0/4188313504 >> v2:172.21.15.157:3300/0 conn(0x7fdccc1444f0 0x7fdccc0d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053374411 0 0) 0x7fdccc0b5a30 con 0x7fdccc0b5570 2024-09-13T16:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.780+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb00035d0 con 0x7fdccc0b5570 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2124630375 0 0) 0x7fdcb00035d0 con 0x7fdccc0b5570 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0d0680 msgr2=0x7fdccc15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0d0680 0x7fdccc15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 >> v2:172.21.15.157:3300/0 conn(0x7fdccc1444f0 msgr2=0x7fdccc0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 --2- 172.21.15.3:0/4188313504 >> v2:172.21.15.157:3300/0 conn(0x7fdccc1444f0 0x7fdccc0d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc0cd890 con 0x7fdccc0b5570 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fdcc40019f0 con 0x7fdccc0b5570 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 >> v1:172.21.15.3:6789/0 conn(0x7fdccc0b5570 legacy=0x7fdccc15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb0003a20 0x7fdcb0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0004870 0x7fdcb0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb0005230 0x7fdcb000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0b5d10 con 0x7fdcb0003a20 2024-09-13T16:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0cd5b0 con 0x7fdcb0005230 2024-09-13T16:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb000bf70 con 0x7fdcb0004870 2024-09-13T16:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccb400700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb0003a20 0x7fdcb0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccb400700 1 -- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0004870 msgr2=0x7fdcb0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccaa00700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0004870 0x7fdcb0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccb400700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0004870 0x7fdcb0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccbe00700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb0005230 0x7fdcb000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccb400700 1 -- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb0005230 msgr2=0x7fdcb000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:01.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccb400700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb0005230 0x7fdcb000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccb400700 1 -- 172.21.15.3:0/4188313504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb00035d0 con 0x7fdcb0003a20 2024-09-13T16:52:01.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccaa00700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0004870 0x7fdcb0004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:01.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccbe00700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb0005230 0x7fdcb000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:01.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccb400700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb0003a20 0x7fdcb0003e40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009ba0 tx=0x7fdcb00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd163998a7e91270 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:01.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00e6c0 con 0x7fdcb0003a20 2024-09-13T16:52:01.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc00e820 con 0x7fdcb0003a20 2024-09-13T16:52:01.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcca000700 1 -- 172.21.15.3:0/4188313504 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00eb90 con 0x7fdcb0003a20 2024-09-13T16:52:01.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcd0a00700 1 -- 172.21.15.3:0/4188313504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb0003a20 msgr2=0x7fdcb0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:01.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/4188313504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb0003a20 0x7fdcb0003e40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009ba0 tx=0x7fdcb00058a0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.784+0000 7fdccbe00700 1 -- 172.21.15.3:0/4188313504 reap_dead start 2024-09-13T16:52:01.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.788+0000 7fdcd0a00700 1 -- 172.21.15.3:0/4188313504 shutdown_connections 2024-09-13T16:52:01.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.788+0000 7fdcd0a00700 1 -- 172.21.15.3:0/4188313504 >> 172.21.15.3:0/4188313504 conn(0x7fdccc0721a0 msgr2=0x7fdccc06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:01.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.788+0000 7fdcd0a00700 1 -- 172.21.15.3:0/4188313504 shutdown_connections 2024-09-13T16:52:01.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.788+0000 7fdcd0a00700 1 -- 172.21.15.3:0/4188313504 wait complete. 2024-09-13T16:52:01.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.788+0000 7fdcd0a00700 1 Processor -- start 2024-09-13T16:52:01.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdcd0a00700 1 -- start start 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdcd0a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0003a20 0x7fdccc1f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdcd0a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc15bb60 0x7fdccc0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdcd0a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0c91b0 0x7fdccc0a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdcd0a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0c9730 con 0x7fdccc15bb60 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdcd0a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc15bfe0 con 0x7fdccc0c91b0 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdcd0a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0ca9e0 con 0x7fdcb0003a20 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc15bb60 0x7fdccc0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc15bb60 0x7fdccc0c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43496/0 (socket says 172.21.15.3:43496) 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 -- 172.21.15.3:0/2812449293 learned_addr learned my addr 172.21.15.3:0/2812449293 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccb400700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0003a20 0x7fdccc1f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccbe00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0c91b0 0x7fdccc0a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 -- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0003a20 msgr2=0x7fdccc1f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0003a20 0x7fdccc1f05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 -- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0c91b0 msgr2=0x7fdccc0a8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0c91b0 0x7fdccc0a8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 -- 172.21.15.3:0/2812449293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc0cacc0 con 0x7fdccc15bb60 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.792+0000 7fdccaa00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc15bb60 0x7fdccc0c8c70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8010900 tx=0x7fdcb8010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.796+0000 7fdcc3400700 1 -- 172.21.15.3:0/2812449293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8004220 con 0x7fdccc15bb60 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.796+0000 7fdcc3400700 1 -- 172.21.15.3:0/2812449293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb8004380 con 0x7fdccc15bb60 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.796+0000 7fdcc3400700 1 -- 172.21.15.3:0/2812449293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80036c0 con 0x7fdccc15bb60 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.796+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc0cafa0 con 0x7fdccc15bb60 2024-09-13T16:52:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.796+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc0c7fe0 con 0x7fdccc15bb60 2024-09-13T16:52:01.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.796+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc0d0140 con 0x7fdccc15bb60 2024-09-13T16:52:01.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.800+0000 7fdcc3400700 1 -- 172.21.15.3:0/2812449293 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdcb8003820 con 0x7fdccc15bb60 2024-09-13T16:52:01.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.800+0000 7fdcc3400700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac06ab40 0x7fdcac06d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:01.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.800+0000 7fdcc3400700 1 -- 172.21.15.3:0/2812449293 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fdcb8084110 con 0x7fdccc15bb60 2024-09-13T16:52:01.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.800+0000 7fdccb400700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac06ab40 0x7fdcac06d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:01.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.800+0000 7fdccb400700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac06ab40 0x7fdcac06d000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0090b0 tx=0x7fdcbc01e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:01.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.808+0000 7fdcc3400700 1 -- 172.21.15.3:0/2812449293 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdcb8051490 con 0x7fdccc15bb60 2024-09-13T16:52:01.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:01.984+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdccc0cab20 con 0x7fdccc15bb60 2024-09-13T16:52:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.568+0000 7fdcc3400700 1 -- 172.21.15.3:0/2812449293 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdccc0cab20 con 0x7fdccc15bb60 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac06ab40 msgr2=0x7fdcac06d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac06ab40 0x7fdcac06d000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0090b0 tx=0x7fdcbc01e040 comp rx=0 tx=0).stop 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc15bb60 msgr2=0x7fdccc0c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc15bb60 0x7fdccc0c8c70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8010900 tx=0x7fdcb8010c10 comp rx=0 tx=0).stop 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 shutdown_connections 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdcac06ab40 0x7fdcac06d000 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0c91b0 0x7fdccc0a8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc15bb60 0x7fdccc0c8c70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 --2- 172.21.15.3:0/2812449293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb0003a20 0x7fdccc1f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.580+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 >> 172.21.15.3:0/2812449293 conn(0x7fdccc0721a0 msgr2=0x7fdccc15e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.584+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 shutdown_connections 2024-09-13T16:52:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.584+0000 7fdcd0a00700 1 -- 172.21.15.3:0/2812449293 wait complete. 2024-09-13T16:52:02.595 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-13T16:52:02.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T16:52:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.792+0000 7f41c9e00700 1 Processor -- start 2024-09-13T16:52:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.792+0000 7f41c9e00700 1 -- start start 2024-09-13T16:52:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c9e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41c4071630 0x7f41c4071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c9e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41c415d260 0x7f41c415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c9e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c415e200 con 0x7f41c4146670 2024-09-13T16:52:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c9e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c40b3e40 con 0x7f41c415d260 2024-09-13T16:52:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c9e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c4072000 con 0x7f41c4071630 2024-09-13T16:52:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c3e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41c4071630 0x7f41c4071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c9400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41c415d260 0x7f41c415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c8a00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f41c4146670 0x7f41c40b3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:40076/0 (socket says 172.21.15.3:40076) 2024-09-13T16:52:02.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c8a00700 1 -- 172.21.15.3:0/1801242461 learned_addr learned my addr 172.21.15.3:0/1801242461 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c3400700 1 -- 172.21.15.3:0/1801242461 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392305402 0 0) 0x7f41c415e200 con 0x7f41c4146670 2024-09-13T16:52:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c3400700 1 -- 172.21.15.3:0/1801242461 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41ac0035d0 con 0x7f41c4146670 2024-09-13T16:52:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c3e00700 1 -- 172.21.15.3:0/1801242461 >> v2:172.21.15.3:3300/0 conn(0x7f41c415d260 msgr2=0x7f41c415d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c3e00700 1 --2- 172.21.15.3:0/1801242461 >> v2:172.21.15.3:3300/0 conn(0x7f41c415d260 0x7f41c415d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c3e00700 1 -- 172.21.15.3:0/1801242461 >> v1:172.21.15.157:6789/0 conn(0x7f41c4146670 legacy=0x7f41c40b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:02.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c3400700 1 -- 172.21.15.3:0/1801242461 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3184187376 0 0) 0x7f41ac0035d0 con 0x7f41c4146670 2024-09-13T16:52:02.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c3e00700 1 -- 172.21.15.3:0/1801242461 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c415df20 con 0x7f41c4071630 2024-09-13T16:52:02.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.796+0000 7f41c9400700 1 --2- 172.21.15.3:0/1801242461 >> v2:172.21.15.3:3300/0 conn(0x7f41c415d260 0x7f41c415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:02.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c3e00700 1 --2- 172.21.15.3:0/1801242461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 0x7f41c4071ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f41c4070ed0 tx=0x7f41b0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce827c58f75ce10b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:02.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c3400700 1 -- 172.21.15.3:0/1801242461 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8001670 con 0x7f41c4071630 2024-09-13T16:52:02.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c3400700 1 -- 172.21.15.3:0/1801242461 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b0004030 con 0x7f41c4071630 2024-09-13T16:52:02.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c3400700 1 -- 172.21.15.3:0/1801242461 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b00043c0 con 0x7f41c4071630 2024-09-13T16:52:02.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c9e00700 1 -- 172.21.15.3:0/1801242461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 msgr2=0x7f41c4071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:02.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1801242461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 0x7f41c4071ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f41c4070ed0 tx=0x7f41b0009b30 comp rx=0 tx=0).stop 2024-09-13T16:52:02.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c9e00700 1 -- 172.21.15.3:0/1801242461 shutdown_connections 2024-09-13T16:52:02.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1801242461 >> v2:172.21.15.3:3300/0 conn(0x7f41c415d260 0x7f41c415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1801242461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 0x7f41c4071ac0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c9e00700 1 -- 172.21.15.3:0/1801242461 >> 172.21.15.3:0/1801242461 conn(0x7f41c40cd380 msgr2=0x7f41c4070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:02.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c9e00700 1 -- 172.21.15.3:0/1801242461 shutdown_connections 2024-09-13T16:52:02.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.800+0000 7f41c9e00700 1 -- 172.21.15.3:0/1801242461 wait complete. 2024-09-13T16:52:02.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.804+0000 7f41c9e00700 1 Processor -- start 2024-09-13T16:52:02.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.804+0000 7f41c9e00700 1 -- start start 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.804+0000 7f41c9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 0x7f41c41f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41c4146670 0x7f41c40cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41c415d260 0x7f41c415c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40aaf30 con 0x7f41c4146670 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40aac50 con 0x7f41c4071630 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40c8770 con 0x7f41c415d260 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41c4146670 0x7f41c40cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41c4146670 0x7f41c40cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43504/0 (socket says 172.21.15.3:43504) 2024-09-13T16:52:02.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 -- 172.21.15.3:0/1206129126 learned_addr learned my addr 172.21.15.3:0/1206129126 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c8a00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 0x7f41c41f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9400700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41c415d260 0x7f41c415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 -- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41c415d260 msgr2=0x7f41c415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41c415d260 0x7f41c415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 -- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 msgr2=0x7f41c41f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 0x7f41c41f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 -- 172.21.15.3:0/1206129126 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc009a70 con 0x7f41c4146670 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c8a00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 0x7f41c41f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:02.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c3e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41c4146670 0x7f41c40cb610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f41b0000c00 tx=0x7f41b0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:02.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c0c00700 1 -- 172.21.15.3:0/1206129126 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0003db0 con 0x7f41c4146670 2024-09-13T16:52:02.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c0c00700 1 -- 172.21.15.3:0/1206129126 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b001f070 con 0x7f41c4146670 2024-09-13T16:52:02.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c0c00700 1 -- 172.21.15.3:0/1206129126 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0011830 con 0x7f41c4146670 2024-09-13T16:52:02.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b00094b0 con 0x7f41c4146670 2024-09-13T16:52:02.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c40c8d30 con 0x7f41c4146670 2024-09-13T16:52:02.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.808+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c4066f50 con 0x7f41c4146670 2024-09-13T16:52:02.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.812+0000 7f41c0c00700 1 -- 172.21.15.3:0/1206129126 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41b0011990 con 0x7f41c4146670 2024-09-13T16:52:02.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.816+0000 7f41c0c00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f41a4066760 0x7f41a4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:02.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.816+0000 7f41c0c00700 1 -- 172.21.15.3:0/1206129126 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f41b0021070 con 0x7f41c4146670 2024-09-13T16:52:02.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.816+0000 7f41c8a00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f41a4066760 0x7f41a4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.820+0000 7f41c8a00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f41a4066760 0x7f41a4068c20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41b8001eb0 tx=0x7f41b8001d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:02.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:02.824+0000 7f41c0c00700 1 -- 172.21.15.3:0/1206129126 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f41b00530d0 con 0x7f41c4146670 2024-09-13T16:52:03.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.044+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f41c40c85d0 con 0x7f41c4146670 2024-09-13T16:52:03.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.056+0000 7f41c0c00700 1 -- 172.21.15.3:0/1206129126 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f41c40c85d0 con 0x7f41c4146670 2024-09-13T16:52:03.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.064+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f41a4066760 msgr2=0x7f41a4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.064+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f41a4066760 0x7f41a4068c20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41b8001eb0 tx=0x7f41b8001d20 comp rx=0 tx=0).stop 2024-09-13T16:52:03.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.064+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41c4146670 msgr2=0x7f41c40cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.064+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41c4146670 0x7f41c40cb610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f41b0000c00 tx=0x7f41b0004770 comp rx=0 tx=0).stop 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.068+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 shutdown_connections 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.068+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f41a4066760 0x7f41a4068c20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.068+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41c415d260 0x7f41c415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.068+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41c4146670 0x7f41c40cb610 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.068+0000 7f41c9e00700 1 --2- 172.21.15.3:0/1206129126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41c4071630 0x7f41c41f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.068+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 >> 172.21.15.3:0/1206129126 conn(0x7f41c40cd380 msgr2=0x7f41c415e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.068+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 shutdown_connections 2024-09-13T16:52:03.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.068+0000 7f41c9e00700 1 -- 172.21.15.3:0/1206129126 wait complete. 2024-09-13T16:52:03.076 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 5 and data pool 6 2024-09-13T16:52:03.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:03.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.284+0000 7f6d0ce00700 1 Processor -- start 2024-09-13T16:52:03.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.284+0000 7f6d0ce00700 1 -- start start 2024-09-13T16:52:03.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d0ce00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6d08159d30 0x7f6d0815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d0ce00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080cdda0 con 0x7f6d08159950 2024-09-13T16:52:03.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d0ce00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080cdff0 con 0x7f6d080721a0 2024-09-13T16:52:03.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d0ce00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080ce990 con 0x7f6d08159d30 2024-09-13T16:52:03.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d05e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d08159950 0x7f6d080b3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38058/0 (socket says 172.21.15.3:38058) 2024-09-13T16:52:03.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d05e00700 1 -- 172.21.15.3:0/651918474 learned_addr learned my addr 172.21.15.3:0/651918474 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:03.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d05400700 1 --2- 172.21.15.3:0/651918474 >> v2:172.21.15.157:3300/0 conn(0x7f6d08159d30 0x7f6d0815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d04a00700 1 -- 172.21.15.3:0/651918474 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428493196 0 0) 0x7f6d080cdda0 con 0x7f6d08159950 2024-09-13T16:52:03.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d04a00700 1 -- 172.21.15.3:0/651918474 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cec0057e0 con 0x7f6d08159950 2024-09-13T16:52:03.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d05400700 1 -- 172.21.15.3:0/651918474 >> v1:172.21.15.157:6789/0 conn(0x7f6d080721a0 legacy=0x7f6d080b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d05400700 1 -- 172.21.15.3:0/651918474 >> v1:172.21.15.3:6789/0 conn(0x7f6d08159950 legacy=0x7f6d080b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.288+0000 7f6d05400700 1 -- 172.21.15.3:0/651918474 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080ce2d0 con 0x7f6d08159d30 2024-09-13T16:52:03.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d05400700 1 --2- 172.21.15.3:0/651918474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 0x7f6d0815d6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009850 tx=0x7f6cf4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9cea30c5364a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:03.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d04a00700 1 -- 172.21.15.3:0/651918474 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4011830 con 0x7f6d08159d30 2024-09-13T16:52:03.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d04a00700 1 -- 172.21.15.3:0/651918474 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf4011990 con 0x7f6d08159d30 2024-09-13T16:52:03.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d04a00700 1 -- 172.21.15.3:0/651918474 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4011d40 con 0x7f6d08159d30 2024-09-13T16:52:03.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d0ce00700 1 -- 172.21.15.3:0/651918474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 msgr2=0x7f6d0815d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d0ce00700 1 --2- 172.21.15.3:0/651918474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 0x7f6d0815d6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009850 tx=0x7f6cf4009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d0ce00700 1 -- 172.21.15.3:0/651918474 shutdown_connections 2024-09-13T16:52:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d0ce00700 1 --2- 172.21.15.3:0/651918474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 0x7f6d0815d6d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d0ce00700 1 -- 172.21.15.3:0/651918474 >> 172.21.15.3:0/651918474 conn(0x7f6d0806a390 msgr2=0x7f6d0806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d0ce00700 1 -- 172.21.15.3:0/651918474 shutdown_connections 2024-09-13T16:52:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.292+0000 7f6d0ce00700 1 -- 172.21.15.3:0/651918474 wait complete. 2024-09-13T16:52:03.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d0ce00700 1 Processor -- start 2024-09-13T16:52:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d0ce00700 1 -- start start 2024-09-13T16:52:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d0ce00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d080721a0 0x7f6d081587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d0ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d08159950 0x7f6d0814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d0ce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 0x7f6d0814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d0ce00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08144770 con 0x7f6d08159950 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d0ce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08144490 con 0x7f6d08159d30 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d0ce00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0815af70 con 0x7f6d080721a0 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d05400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d08159950 0x7f6d0814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d05400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d08159950 0x7f6d0814d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43508/0 (socket says 172.21.15.3:43508) 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d06800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 0x7f6d0814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d05400700 1 -- 172.21.15.3:0/3470402146 learned_addr learned my addr 172.21.15.3:0/3470402146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.296+0000 7f6d05e00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d080721a0 0x7f6d081587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d05400700 1 -- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d080721a0 msgr2=0x7f6d081587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d05400700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d080721a0 0x7f6d081587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d05400700 1 -- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 msgr2=0x7f6d0814f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d05400700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 0x7f6d0814f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d05400700 1 -- 172.21.15.3:0/3470402146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40094b0 con 0x7f6d08159950 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d06800700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 0x7f6d0814f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d05400700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d08159950 0x7f6d0814d860 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4014040 tx=0x7f6cf4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6cfe000700 1 -- 172.21.15.3:0/3470402146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4019980 con 0x7f6d08159950 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6cfe000700 1 -- 172.21.15.3:0/3470402146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf401e070 con 0x7f6d08159950 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6cfe000700 1 -- 172.21.15.3:0/3470402146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4019d20 con 0x7f6d08159950 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0815b250 con 0x7f6d08159950 2024-09-13T16:52:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d08158d30 con 0x7f6d08159950 2024-09-13T16:52:03.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.300+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0815b770 con 0x7f6d08159950 2024-09-13T16:52:03.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.304+0000 7f6cfe000700 1 -- 172.21.15.3:0/3470402146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6cf402d450 con 0x7f6d08159950 2024-09-13T16:52:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.304+0000 7f6cfe000700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6ce8066730 0x7f6ce8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.304+0000 7f6cfe000700 1 -- 172.21.15.3:0/3470402146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6cf4085230 con 0x7f6d08159950 2024-09-13T16:52:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.304+0000 7f6d05e00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6ce8066730 0x7f6ce8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.308+0000 7f6d05e00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6ce8066730 0x7f6ce8068bf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8007fd0 tx=0x7f6cf8007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.312+0000 7f6cfe000700 1 -- 172.21.15.3:0/3470402146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6d0815b770 con 0x7f6d08159950 2024-09-13T16:52:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.500+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d0815b770 con 0x7f6d08159950 2024-09-13T16:52:03.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.500+0000 7f6cfe000700 1 -- 172.21.15.3:0/3470402146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16739 (secure 0 0 0) 0x7f6d0815b770 con 0x7f6d08159950 2024-09-13T16:52:03.509 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:03.509 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:03.048382+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:01.009263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:02.059017+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:03.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6ce8066730 msgr2=0x7f6ce8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6ce8066730 0x7f6ce8068bf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8007fd0 tx=0x7f6cf8007de0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d08159950 msgr2=0x7f6d0814d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d08159950 0x7f6d0814d860 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4014040 tx=0x7f6cf4004770 comp rx=0 tx=0).stop 2024-09-13T16:52:03.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 shutdown_connections 2024-09-13T16:52:03.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6ce8066730 0x7f6ce8068bf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d08159d30 0x7f6d0814f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d08159950 0x7f6d0814d860 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 --2- 172.21.15.3:0/3470402146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d080721a0 0x7f6d081587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 >> 172.21.15.3:0/3470402146 conn(0x7f6d0806a390 msgr2=0x7f6d0814c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:03.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 shutdown_connections 2024-09-13T16:52:03.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.504+0000 7f6d0ce00700 1 -- 172.21.15.3:0/3470402146 wait complete. 2024-09-13T16:52:03.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:03.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.704+0000 7f0cf8a00700 1 Processor -- start 2024-09-13T16:52:03.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.704+0000 7f0cf8a00700 1 -- start start 2024-09-13T16:52:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf8a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cf40ce3b0 0x7f0cf415dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf8a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0cf40720c0 0x7f0cf406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf8a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40724f0 con 0x7f0cf406d0e0 2024-09-13T16:52:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf8a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf406da90 con 0x7f0cf40ce3b0 2024-09-13T16:52:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf8a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf40b2c90 con 0x7f0cf40720c0 2024-09-13T16:52:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf1e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cf40ce3b0 0x7f0cf415dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf2800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0cf40720c0 0x7f0cf406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf1e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cf40ce3b0 0x7f0cf415dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44840/0 (socket says 172.21.15.3:44840) 2024-09-13T16:52:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf1e00700 1 -- 172.21.15.3:0/1008854962 learned_addr learned my addr 172.21.15.3:0/1008854962 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf2800700 1 -- 172.21.15.3:0/1008854962 >> v2:172.21.15.3:3300/0 conn(0x7f0cf40ce3b0 msgr2=0x7f0cf415dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf2800700 1 --2- 172.21.15.3:0/1008854962 >> v2:172.21.15.3:3300/0 conn(0x7f0cf40ce3b0 0x7f0cf415dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf2800700 1 -- 172.21.15.3:0/1008854962 >> v1:172.21.15.157:6789/0 conn(0x7f0cf406d0e0 legacy=0x7f0cf4160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf2800700 1 -- 172.21.15.3:0/1008854962 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf415e320 con 0x7f0cf40720c0 2024-09-13T16:52:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf1e00700 1 --2- 172.21.15.3:0/1008854962 >> v2:172.21.15.3:3300/0 conn(0x7f0cf40ce3b0 0x7f0cf415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf0a00700 1 -- 172.21.15.3:0/1008854962 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185352988 0 0) 0x7f0cf40724f0 con 0x7f0cf406d0e0 2024-09-13T16:52:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.708+0000 7f0cf2800700 1 --2- 172.21.15.3:0/1008854962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 0x7f0cf406d550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0cf415d360 tx=0x7f0cec009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=805cabf106c855e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:03.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf0a00700 1 -- 172.21.15.3:0/1008854962 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec011830 con 0x7f0cf40720c0 2024-09-13T16:52:03.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf0a00700 1 -- 172.21.15.3:0/1008854962 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec011990 con 0x7f0cf40720c0 2024-09-13T16:52:03.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf0a00700 1 -- 172.21.15.3:0/1008854962 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec011d20 con 0x7f0cf40720c0 2024-09-13T16:52:03.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf8a00700 1 -- 172.21.15.3:0/1008854962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 msgr2=0x7f0cf406d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/1008854962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 0x7f0cf406d550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0cf415d360 tx=0x7f0cec009b30 comp rx=0 tx=0).stop 2024-09-13T16:52:03.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf8a00700 1 -- 172.21.15.3:0/1008854962 shutdown_connections 2024-09-13T16:52:03.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/1008854962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 0x7f0cf406d550 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/1008854962 >> v2:172.21.15.3:3300/0 conn(0x7f0cf40ce3b0 0x7f0cf415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf8a00700 1 -- 172.21.15.3:0/1008854962 >> 172.21.15.3:0/1008854962 conn(0x7f0cf406a390 msgr2=0x7f0cf406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:03.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf8a00700 1 -- 172.21.15.3:0/1008854962 shutdown_connections 2024-09-13T16:52:03.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.712+0000 7f0cf8a00700 1 -- 172.21.15.3:0/1008854962 wait complete. 2024-09-13T16:52:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf8a00700 1 Processor -- start 2024-09-13T16:52:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf8a00700 1 -- start start 2024-09-13T16:52:03.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf8a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf406d0e0 0x7f0cf41599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf8a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 0x7f0cf414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf8a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cf40ce3b0 0x7f0cf415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf8a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4159f30 con 0x7f0cf406d0e0 2024-09-13T16:52:03.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf8a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40afcb0 con 0x7f0cf40720c0 2024-09-13T16:52:03.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf8a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40aedf0 con 0x7f0cf40ce3b0 2024-09-13T16:52:03.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf406d0e0 0x7f0cf41599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf406d0e0 0x7f0cf41599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44842/0 (socket says 172.21.15.3:44842) 2024-09-13T16:52:03.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 -- 172.21.15.3:0/3026407376 learned_addr learned my addr 172.21.15.3:0/3026407376 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:03.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1400700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 0x7f0cf414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf2800700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cf40ce3b0 0x7f0cf415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 -- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cf40ce3b0 msgr2=0x7f0cf415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cf40ce3b0 0x7f0cf415f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 -- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 msgr2=0x7f0cf414c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 0x7f0cf414c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 -- 172.21.15.3:0/3026407376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4009b30 con 0x7f0cf406d0e0 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1400700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 0x7f0cf414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf2800700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cf40ce3b0 0x7f0cf415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:03.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.716+0000 7f0cf1e00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf406d0e0 0x7f0cf41599f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400efc0 tx=0x7f0ce400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:03.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.720+0000 7f0cea000700 1 -- 172.21.15.3:0/3026407376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400cc70 con 0x7f0cf406d0e0 2024-09-13T16:52:03.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.720+0000 7f0cea000700 1 -- 172.21.15.3:0/3026407376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce400f040 con 0x7f0cf406d0e0 2024-09-13T16:52:03.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.720+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cec0094b0 con 0x7f0cf406d0e0 2024-09-13T16:52:03.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.720+0000 7f0cea000700 1 -- 172.21.15.3:0/3026407376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4014480 con 0x7f0cf406d0e0 2024-09-13T16:52:03.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.720+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf40af3b0 con 0x7f0cf406d0e0 2024-09-13T16:52:03.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.720+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf40b1d70 con 0x7f0cf406d0e0 2024-09-13T16:52:03.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.720+0000 7f0cea000700 1 -- 172.21.15.3:0/3026407376 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ce40145e0 con 0x7f0cf406d0e0 2024-09-13T16:52:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.724+0000 7f0cea000700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0cd4066730 0x7f0cd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.724+0000 7f0cea000700 1 -- 172.21.15.3:0/3026407376 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0ce4084830 con 0x7f0cf406d0e0 2024-09-13T16:52:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.724+0000 7f0cf1400700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0cd4066730 0x7f0cd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.724+0000 7f0cf1400700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0cd4066730 0x7f0cd4068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40b0580 tx=0x7f0ce0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.728+0000 7f0cea000700 1 -- 172.21.15.3:0/3026407376 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ce4051af0 con 0x7f0cf406d0e0 2024-09-13T16:52:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.920+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cf40b3900 con 0x7f0cf406d0e0 2024-09-13T16:52:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.924+0000 7f0cea000700 1 -- 172.21.15.3:0/3026407376 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16739 (secure 0 0 0) 0x7f0ce4055110 con 0x7f0cf406d0e0 2024-09-13T16:52:03.929 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:03.930 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:03.048382+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:01.009263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:02.059017+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0cd4066730 msgr2=0x7f0cd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0cd4066730 0x7f0cd4068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40b0580 tx=0x7f0ce0009040 comp rx=0 tx=0).stop 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf406d0e0 msgr2=0x7f0cf41599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf406d0e0 0x7f0cf41599f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400efc0 tx=0x7f0ce400c3a0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 shutdown_connections 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0cd4066730 0x7f0cd4068bf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cf40ce3b0 0x7f0cf415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf40720c0 0x7f0cf414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 --2- 172.21.15.3:0/3026407376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf406d0e0 0x7f0cf41599f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 >> 172.21.15.3:0/3026407376 conn(0x7f0cf406a390 msgr2=0x7f0cf406b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 shutdown_connections 2024-09-13T16:52:03.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:03.928+0000 7f0cf8a00700 1 -- 172.21.15.3:0/3026407376 wait complete. 2024-09-13T16:52:03.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T16:52:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.136+0000 7fd209600700 1 Processor -- start 2024-09-13T16:52:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd209600700 1 -- start start 2024-09-13T16:52:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd209600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd2040b49d0 0x7fd2040b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd209600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2040b5360 con 0x7fd2040cea10 2024-09-13T16:52:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd209600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2040cef10 con 0x7fd204146050 2024-09-13T16:52:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd209600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd204146430 con 0x7fd2040b49d0 2024-09-13T16:52:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd208c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd2040b49d0 0x7fd2040b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd208c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd2040b49d0 0x7fd2040b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44854/0 (socket says 172.21.15.3:44854) 2024-09-13T16:52:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd208c00700 1 -- 172.21.15.3:0/310982046 learned_addr learned my addr 172.21.15.3:0/310982046 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:04.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd208c00700 1 -- 172.21.15.3:0/310982046 >> v1:172.21.15.157:6789/0 conn(0x7fd204146050 legacy=0x7fd20406ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:04.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd208c00700 1 -- 172.21.15.3:0/310982046 >> v1:172.21.15.3:6789/0 conn(0x7fd2040cea10 legacy=0x7fd20415d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:04.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.140+0000 7fd208c00700 1 -- 172.21.15.3:0/310982046 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040cf1f0 con 0x7fd2040b49d0 2024-09-13T16:52:04.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd208c00700 1 --2- 172.21.15.3:0/310982046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040b49d0 0x7fd2040b4e20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009e50 tx=0x7fd1fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d54c7a55cabc614 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:04.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd202a00700 1 -- 172.21.15.3:0/310982046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00e070 con 0x7fd2040b49d0 2024-09-13T16:52:04.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd202a00700 1 -- 172.21.15.3:0/310982046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc003a40 con 0x7fd2040b49d0 2024-09-13T16:52:04.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd202a00700 1 -- 172.21.15.3:0/310982046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc003df0 con 0x7fd2040b49d0 2024-09-13T16:52:04.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd209600700 1 -- 172.21.15.3:0/310982046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040b49d0 msgr2=0x7fd2040b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:04.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd209600700 1 --2- 172.21.15.3:0/310982046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040b49d0 0x7fd2040b4e20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009e50 tx=0x7fd1fc00b330 comp rx=0 tx=0).stop 2024-09-13T16:52:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd209600700 1 -- 172.21.15.3:0/310982046 shutdown_connections 2024-09-13T16:52:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd209600700 1 --2- 172.21.15.3:0/310982046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040b49d0 0x7fd2040b4e20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd209600700 1 -- 172.21.15.3:0/310982046 >> 172.21.15.3:0/310982046 conn(0x7fd2040721a0 msgr2=0x7fd20406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd209600700 1 -- 172.21.15.3:0/310982046 shutdown_connections 2024-09-13T16:52:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.144+0000 7fd209600700 1 -- 172.21.15.3:0/310982046 wait complete. 2024-09-13T16:52:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 Processor -- start 2024-09-13T16:52:04.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 -- start start 2024-09-13T16:52:04.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040b49d0 0x7fd204144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:04.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040cea10 0x7fd2040a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:04.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204146050 0x7fd2040aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:04.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040cafe0 con 0x7fd2040cea10 2024-09-13T16:52:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040cb2c0 con 0x7fd2040b49d0 2024-09-13T16:52:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040cb5a0 con 0x7fd204146050 2024-09-13T16:52:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040cea10 0x7fd2040a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040cea10 0x7fd2040a84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44864/0 (socket says 172.21.15.3:44864) 2024-09-13T16:52:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 -- 172.21.15.3:0/3239795671 learned_addr learned my addr 172.21.15.3:0/3239795671 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203e00700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040b49d0 0x7fd204144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd208c00700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204146050 0x7fd2040aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 -- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204146050 msgr2=0x7fd2040aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204146050 0x7fd2040aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 -- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040b49d0 msgr2=0x7fd204144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040b49d0 0x7fd204144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 -- 172.21.15.3:0/3239795671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc009b30 con 0x7fd2040cea10 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd208c00700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204146050 0x7fd2040aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd203400700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040cea10 0x7fd2040a84e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000c450 tx=0x7fd1f000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd1fbe00700 1 -- 172.21.15.3:0/3239795671 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00052d0 con 0x7fd2040cea10 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd1fbe00700 1 -- 172.21.15.3:0/3239795671 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f0005430 con 0x7fd2040cea10 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd1fbe00700 1 -- 172.21.15.3:0/3239795671 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00055a0 con 0x7fd2040cea10 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.148+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2040cb880 con 0x7fd2040cea10 2024-09-13T16:52:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.152+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2040a8a20 con 0x7fd2040cea10 2024-09-13T16:52:04.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.152+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2040b4e20 con 0x7fd2040cea10 2024-09-13T16:52:04.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.152+0000 7fd1fbe00700 1 -- 172.21.15.3:0/3239795671 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd1f00036a0 con 0x7fd2040cea10 2024-09-13T16:52:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.156+0000 7fd1fbe00700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd1e4066760 0x7fd1e4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.156+0000 7fd1fbe00700 1 -- 172.21.15.3:0/3239795671 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd1f0083440 con 0x7fd2040cea10 2024-09-13T16:52:04.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.156+0000 7fd203e00700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd1e4066760 0x7fd1e4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:04.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.156+0000 7fd203e00700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd1e4066760 0x7fd1e4068c20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4001210 tx=0x7fd1f40011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.160+0000 7fd1fbe00700 1 -- 172.21.15.3:0/3239795671 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd1f0050700 con 0x7fd2040cea10 2024-09-13T16:52:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:04.384+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd2040b3fd0 con 0x7fd2040cea10 2024-09-13T16:52:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.068+0000 7fd1fbe00700 1 -- 172.21.15.3:0/3239795671 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd1f0053d20 con 0x7fd2040cea10 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd1e4066760 msgr2=0x7fd1e4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd1e4066760 0x7fd1e4068c20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4001210 tx=0x7fd1f40011a0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040cea10 msgr2=0x7fd2040a84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040cea10 0x7fd2040a84e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000c450 tx=0x7fd1f000c760 comp rx=0 tx=0).stop 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 shutdown_connections 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd1e4066760 0x7fd1e4068c20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204146050 0x7fd2040aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2040cea10 0x7fd2040a84e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 --2- 172.21.15.3:0/3239795671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040b49d0 0x7fd204144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 >> 172.21.15.3:0/3239795671 conn(0x7fd2040721a0 msgr2=0x7fd2041e7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 shutdown_connections 2024-09-13T16:52:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.072+0000 7fd209600700 1 -- 172.21.15.3:0/3239795671 wait complete. 2024-09-13T16:52:05.105 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T16:52:05.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T16:52:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.284+0000 7f576e200700 1 Processor -- start 2024-09-13T16:52:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.284+0000 7f576e200700 1 -- start start 2024-09-13T16:52:05.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576e200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f576815ac50 0x7f57680cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:05.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576e200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f576806a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:05.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576e200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57680afe90 con 0x7f57680b4fc0 2024-09-13T16:52:05.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576e200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57680cfe80 con 0x7f576815ac50 2024-09-13T16:52:05.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576e200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57680af880 con 0x7f576815b740 2024-09-13T16:52:05.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576d800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f576806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:05.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576ce00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f576815ac50 0x7f57680cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:05.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f5767e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f57680b4fc0 0x7f576815b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:35910/0 (socket says 172.21.15.3:35910) 2024-09-13T16:52:05.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f5767e00700 1 -- 172.21.15.3:0/3489453715 learned_addr learned my addr 172.21.15.3:0/3489453715 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:05.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576d800700 1 -- 172.21.15.3:0/3489453715 >> v2:172.21.15.3:3300/0 conn(0x7f576815ac50 msgr2=0x7f57680cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:05.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576d800700 1 --2- 172.21.15.3:0/3489453715 >> v2:172.21.15.3:3300/0 conn(0x7f576815ac50 0x7f57680cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576d800700 1 -- 172.21.15.3:0/3489453715 >> v1:172.21.15.157:6789/0 conn(0x7f57680b4fc0 legacy=0x7f576815b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:05.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576d800700 1 -- 172.21.15.3:0/3489453715 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57680cf610 con 0x7f576815b740 2024-09-13T16:52:05.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f5767400700 1 -- 172.21.15.3:0/3489453715 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723060751 0 0) 0x7f57680afe90 con 0x7f57680b4fc0 2024-09-13T16:52:05.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.288+0000 7f576d800700 1 --2- 172.21.15.3:0/3489453715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f576806a970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5760009d90 tx=0x7f576000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9277e54230fb6cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:05.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f5767400700 1 -- 172.21.15.3:0/3489453715 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576000e070 con 0x7f576815b740 2024-09-13T16:52:05.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f5767400700 1 -- 172.21.15.3:0/3489453715 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576000bb20 con 0x7f576815b740 2024-09-13T16:52:05.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f5767400700 1 -- 172.21.15.3:0/3489453715 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760012430 con 0x7f576815b740 2024-09-13T16:52:05.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f576e200700 1 -- 172.21.15.3:0/3489453715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 msgr2=0x7f576806a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:05.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f576e200700 1 --2- 172.21.15.3:0/3489453715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f576806a970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5760009d90 tx=0x7f576000b330 comp rx=0 tx=0).stop 2024-09-13T16:52:05.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f576e200700 1 -- 172.21.15.3:0/3489453715 shutdown_connections 2024-09-13T16:52:05.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f576e200700 1 --2- 172.21.15.3:0/3489453715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f576806a970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f576e200700 1 --2- 172.21.15.3:0/3489453715 >> v2:172.21.15.3:3300/0 conn(0x7f576815ac50 0x7f57680cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f576e200700 1 -- 172.21.15.3:0/3489453715 >> 172.21.15.3:0/3489453715 conn(0x7f57680b2d50 msgr2=0x7f57680b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:05.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f576e200700 1 -- 172.21.15.3:0/3489453715 shutdown_connections 2024-09-13T16:52:05.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.292+0000 7f576e200700 1 -- 172.21.15.3:0/3489453715 wait complete. 2024-09-13T16:52:05.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 Processor -- start 2024-09-13T16:52:05.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 -- start start 2024-09-13T16:52:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57680b4fc0 0x7f57681485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576815ac50 0x7f576806ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f57680c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768148b20 con 0x7f57680b4fc0 2024-09-13T16:52:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576809bad0 con 0x7f576815ac50 2024-09-13T16:52:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576806e410 con 0x7f576815b740 2024-09-13T16:52:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57680b4fc0 0x7f57681485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:05.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57680b4fc0 0x7f57681485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44884/0 (socket says 172.21.15.3:44884) 2024-09-13T16:52:05.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 -- 172.21.15.3:0/3307053493 learned_addr learned my addr 172.21.15.3:0/3307053493 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f5767e00700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576815ac50 0x7f576806ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576d800700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f57680c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 -- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 msgr2=0x7f57680c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f57680c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 -- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576815ac50 msgr2=0x7f576806ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576815ac50 0x7f576806ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 -- 172.21.15.3:0/3307053493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760009a70 con 0x7f57680b4fc0 2024-09-13T16:52:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f5767e00700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576815ac50 0x7f576806ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576ce00700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57680b4fc0 0x7f57681485e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f575c00e9d0 tx=0x7f575c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f5764c00700 1 -- 172.21.15.3:0/3307053493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c009720 con 0x7f57680b4fc0 2024-09-13T16:52:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f5764c00700 1 -- 172.21.15.3:0/3307053493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c00c850 con 0x7f57680b4fc0 2024-09-13T16:52:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576809aec0 con 0x7f57680b4fc0 2024-09-13T16:52:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f5764c00700 1 -- 172.21.15.3:0/3307053493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c00cbc0 con 0x7f57680b4fc0 2024-09-13T16:52:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.296+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576809b480 con 0x7f57680b4fc0 2024-09-13T16:52:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.300+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576806b1b0 con 0x7f57680b4fc0 2024-09-13T16:52:05.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.300+0000 7f5764c00700 1 -- 172.21.15.3:0/3307053493 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f575c00cd20 con 0x7f57680b4fc0 2024-09-13T16:52:05.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.304+0000 7f5764c00700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5748066730 0x7f5748068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:05.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.304+0000 7f5764c00700 1 -- 172.21.15.3:0/3307053493 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f575c084470 con 0x7f57680b4fc0 2024-09-13T16:52:05.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.304+0000 7f5767e00700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5748066730 0x7f5748068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.304+0000 7f5767e00700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5748066730 0x7f5748068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f57680ceaa0 tx=0x7f5754007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:05.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.312+0000 7f5764c00700 1 -- 172.21.15.3:0/3307053493 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f575c051600 con 0x7f57680b4fc0 2024-09-13T16:52:05.536 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-13T16:52:05.532+0000 7ff908800700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:52:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:05.536+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f57680ceb90 con 0x7f57680b4fc0 2024-09-13T16:52:06.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.076+0000 7f5764c00700 1 -- 172.21.15.3:0/3307053493 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f575c054c20 con 0x7f57680b4fc0 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5748066730 msgr2=0x7f5748068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5748066730 0x7f5748068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f57680ceaa0 tx=0x7f5754007cc0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57680b4fc0 msgr2=0x7f57681485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57680b4fc0 0x7f57681485e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f575c00e9d0 tx=0x7f575c00ed90 comp rx=0 tx=0).stop 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 shutdown_connections 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5748066730 0x7f5748068bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f576815b740 0x7f57680c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576815ac50 0x7f576806ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 --2- 172.21.15.3:0/3307053493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57680b4fc0 0x7f57681485e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 >> 172.21.15.3:0/3307053493 conn(0x7f57680b2d50 msgr2=0x7f57680b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 shutdown_connections 2024-09-13T16:52:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.080+0000 7f576e200700 1 -- 172.21.15.3:0/3307053493 wait complete. 2024-09-13T16:52:06.112 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T16:52:06.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.292+0000 7f5ef7c00700 1 Processor -- start 2024-09-13T16:52:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.296+0000 7f5ef7c00700 1 -- start start 2024-09-13T16:52:06.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.296+0000 7f5ef7c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef0157430 0x7f5ef006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.296+0000 7f5ef7c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef006d870 con 0x7f5ef00cc920 2024-09-13T16:52:06.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.296+0000 7f5ef7c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef006e710 con 0x7f5ef0072280 2024-09-13T16:52:06.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.296+0000 7f5ef7c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef006e9f0 con 0x7f5ef0157430 2024-09-13T16:52:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.296+0000 7f5eefe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5ef00cc920 0x7f5ef0071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45272/0 (socket says 172.21.15.3:45272) 2024-09-13T16:52:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.296+0000 7f5eefe00700 1 -- 172.21.15.3:0/1022552141 learned_addr learned my addr 172.21.15.3:0/1022552141 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.296+0000 7f5ef5200700 1 --2- 172.21.15.3:0/1022552141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef0157430 0x7f5ef006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5eef400700 1 -- 172.21.15.3:0/1022552141 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245777640 0 0) 0x7f5ef006d870 con 0x7f5ef00cc920 2024-09-13T16:52:06.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5eef400700 1 -- 172.21.15.3:0/1022552141 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed80035d0 con 0x7f5ef00cc920 2024-09-13T16:52:06.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5eef400700 1 -- 172.21.15.3:0/1022552141 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832106046 0 0) 0x7f5ef006e710 con 0x7f5ef0072280 2024-09-13T16:52:06.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5eef400700 1 -- 172.21.15.3:0/1022552141 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ef006d870 con 0x7f5ef0072280 2024-09-13T16:52:06.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef5200700 1 -- 172.21.15.3:0/1022552141 >> v1:172.21.15.157:6789/0 conn(0x7f5ef0072280 legacy=0x7f5ef00b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef5200700 1 -- 172.21.15.3:0/1022552141 >> v1:172.21.15.3:6789/0 conn(0x7f5ef00cc920 legacy=0x7f5ef0071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef5200700 1 -- 172.21.15.3:0/1022552141 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef006ecd0 con 0x7f5ef0157430 2024-09-13T16:52:06.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef5200700 1 --2- 172.21.15.3:0/1022552141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef0157430 0x7f5ef006dd00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009880 tx=0x7f5ee4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caf6105c15832f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5eef400700 1 -- 172.21.15.3:0/1022552141 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4011830 con 0x7f5ef0157430 2024-09-13T16:52:06.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5eef400700 1 -- 172.21.15.3:0/1022552141 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee4011990 con 0x7f5ef0157430 2024-09-13T16:52:06.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5eef400700 1 -- 172.21.15.3:0/1022552141 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4011d20 con 0x7f5ef0157430 2024-09-13T16:52:06.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1022552141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef0157430 msgr2=0x7f5ef006dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef7c00700 1 --2- 172.21.15.3:0/1022552141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef0157430 0x7f5ef006dd00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009880 tx=0x7f5ee4009b90 comp rx=0 tx=0).stop 2024-09-13T16:52:06.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1022552141 shutdown_connections 2024-09-13T16:52:06.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef7c00700 1 --2- 172.21.15.3:0/1022552141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef0157430 0x7f5ef006dd00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1022552141 >> 172.21.15.3:0/1022552141 conn(0x7f5ef0071050 msgr2=0x7f5ef0071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:06.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1022552141 shutdown_connections 2024-09-13T16:52:06.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.300+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1022552141 wait complete. 2024-09-13T16:52:06.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef7c00700 1 Processor -- start 2024-09-13T16:52:06.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef7c00700 1 -- start start 2024-09-13T16:52:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef7c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef0072280 0x7f5ef014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef7c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef00cc920 0x7f5ef0159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef7c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0157430 0x7f5ef00ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef7c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0158980 con 0x7f5ef0072280 2024-09-13T16:52:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef7c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef014cd10 con 0x7f5ef0157430 2024-09-13T16:52:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef7c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0157a20 con 0x7f5ef00cc920 2024-09-13T16:52:06.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef5200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef0072280 0x7f5ef014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef5200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef0072280 0x7f5ef014c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44886/0 (socket says 172.21.15.3:44886) 2024-09-13T16:52:06.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.304+0000 7f5ef5200700 1 -- 172.21.15.3:0/1395579836 learned_addr learned my addr 172.21.15.3:0/1395579836 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef5c00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0157430 0x7f5ef00ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5eefe00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef00cc920 0x7f5ef0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef5200700 1 -- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef00cc920 msgr2=0x7f5ef0159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef5200700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef00cc920 0x7f5ef0159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef5200700 1 -- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0157430 msgr2=0x7f5ef00ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef5200700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0157430 0x7f5ef00ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef5200700 1 -- 172.21.15.3:0/1395579836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40094b0 con 0x7f5ef0072280 2024-09-13T16:52:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef5200700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef0072280 0x7f5ef014c7d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4013040 tx=0x7f5ee4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5eecc00700 1 -- 172.21.15.3:0/1395579836 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee401a8c0 con 0x7f5ef0072280 2024-09-13T16:52:06.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5eecc00700 1 -- 172.21.15.3:0/1395579836 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee401e070 con 0x7f5ef0072280 2024-09-13T16:52:06.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef0157d00 con 0x7f5ef0072280 2024-09-13T16:52:06.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5eecc00700 1 -- 172.21.15.3:0/1395579836 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee401ac40 con 0x7f5ef0072280 2024-09-13T16:52:06.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef01582c0 con 0x7f5ef0072280 2024-09-13T16:52:06.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.308+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef00b4ba0 con 0x7f5ef0072280 2024-09-13T16:52:06.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.312+0000 7f5eecc00700 1 -- 172.21.15.3:0/1395579836 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5ee402d450 con 0x7f5ef0072280 2024-09-13T16:52:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.312+0000 7f5eecc00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ed0066730 0x7f5ed0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.312+0000 7f5eecc00700 1 -- 172.21.15.3:0/1395579836 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5ee40851b0 con 0x7f5ef0072280 2024-09-13T16:52:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.316+0000 7f5eefe00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ed0066730 0x7f5ed0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.316+0000 7f5eefe00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ed0066730 0x7f5ed0068bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ef01f5db0 tx=0x7f5edc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.320+0000 7f5eecc00700 1 -- 172.21.15.3:0/1395579836 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5ee40523d0 con 0x7f5ef0072280 2024-09-13T16:52:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.544+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ef0157810 con 0x7f5ef0072280 2024-09-13T16:52:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.544+0000 7f5eecc00700 1 -- 172.21.15.3:0/1395579836 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f5ee40559f0 con 0x7f5ef0072280 2024-09-13T16:52:06.550 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:06.550 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1649229219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1649229219},{"type":"v1","addr":"172.21.15.3:6833","nonce":1649229219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3459985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3459985282},{"type":"v1","addr":"172.21.15.157:6835","nonce":3459985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3779761732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3779761732},{"type":"v1","addr":"172.21.15.157:6837","nonce":3779761732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:06.068076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5373":{"gid":5373,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/750657669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":750657669},{"type":"v1","addr":"172.21.15.3:6835","nonce":750657669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ed0066730 msgr2=0x7f5ed0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ed0066730 0x7f5ed0068bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ef01f5db0 tx=0x7f5edc009040 comp rx=0 tx=0).stop 2024-09-13T16:52:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef0072280 msgr2=0x7f5ef014c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef0072280 0x7f5ef014c7d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4013040 tx=0x7f5ee4004770 comp rx=0 tx=0).stop 2024-09-13T16:52:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 shutdown_connections 2024-09-13T16:52:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5ed0066730 0x7f5ed0068bf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef0157430 0x7f5ef00ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ef00cc920 0x7f5ef0159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 --2- 172.21.15.3:0/1395579836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef0072280 0x7f5ef014c7d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 >> 172.21.15.3:0/1395579836 conn(0x7f5ef0071050 msgr2=0x7f5ef00cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 shutdown_connections 2024-09-13T16:52:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.548+0000 7f5ef7c00700 1 -- 172.21.15.3:0/1395579836 wait complete. 2024-09-13T16:52:06.556 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 16 2024-09-13T16:52:06.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:06.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.732+0000 7ff354000700 1 Processor -- start 2024-09-13T16:52:06.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.736+0000 7ff354000700 1 -- start start 2024-09-13T16:52:06.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.736+0000 7ff354000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff34c160050 0x7ff34c06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.736+0000 7ff354000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff34c06d680 0x7ff34c06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.736+0000 7ff354000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34c06dbf0 con 0x7ff34c160430 2024-09-13T16:52:06.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.736+0000 7ff354000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34c06e870 con 0x7ff34c06d680 2024-09-13T16:52:06.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.736+0000 7ff354000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34c06f1d0 con 0x7ff34c160050 2024-09-13T16:52:06.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff352000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff34c06d680 0x7ff34c06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff351600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff34c160050 0x7ff34c06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff352000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff34c06d680 0x7ff34c06b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44900/0 (socket says 172.21.15.3:44900) 2024-09-13T16:52:06.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff352000700 1 -- 172.21.15.3:0/3148932490 learned_addr learned my addr 172.21.15.3:0/3148932490 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:06.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff352000700 1 -- 172.21.15.3:0/3148932490 >> v2:172.21.15.157:3300/0 conn(0x7ff34c160050 msgr2=0x7ff34c06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:06.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff352000700 1 --2- 172.21.15.3:0/3148932490 >> v2:172.21.15.157:3300/0 conn(0x7ff34c160050 0x7ff34c06ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff352000700 1 -- 172.21.15.3:0/3148932490 >> v1:172.21.15.157:6789/0 conn(0x7ff34c160430 legacy=0x7ff34c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff352000700 1 -- 172.21.15.3:0/3148932490 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c06ded0 con 0x7ff34c06d680 2024-09-13T16:52:06.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff352000700 1 --2- 172.21.15.3:0/3148932490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c06b3a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0717b0 tx=0x7ff34400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae26d07833fae14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff34be00700 1 -- 172.21.15.3:0/3148932490 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34400e070 con 0x7ff34c06d680 2024-09-13T16:52:06.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff34be00700 1 -- 172.21.15.3:0/3148932490 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34400b8d0 con 0x7ff34c06d680 2024-09-13T16:52:06.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff34be00700 1 -- 172.21.15.3:0/3148932490 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34400bc80 con 0x7ff34c06d680 2024-09-13T16:52:06.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff354000700 1 -- 172.21.15.3:0/3148932490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 msgr2=0x7ff34c06b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff354000700 1 --2- 172.21.15.3:0/3148932490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c06b3a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0717b0 tx=0x7ff34400b330 comp rx=0 tx=0).stop 2024-09-13T16:52:06.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff354000700 1 -- 172.21.15.3:0/3148932490 shutdown_connections 2024-09-13T16:52:06.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff354000700 1 --2- 172.21.15.3:0/3148932490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c06b3a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff354000700 1 --2- 172.21.15.3:0/3148932490 >> v2:172.21.15.157:3300/0 conn(0x7ff34c160050 0x7ff34c06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff354000700 1 -- 172.21.15.3:0/3148932490 >> 172.21.15.3:0/3148932490 conn(0x7ff34c0707c0 msgr2=0x7ff34c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:06.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff354000700 1 -- 172.21.15.3:0/3148932490 shutdown_connections 2024-09-13T16:52:06.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.740+0000 7ff354000700 1 -- 172.21.15.3:0/3148932490 wait complete. 2024-09-13T16:52:06.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.744+0000 7ff354000700 1 Processor -- start 2024-09-13T16:52:06.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.744+0000 7ff354000700 1 -- start start 2024-09-13T16:52:06.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.744+0000 7ff354000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c14f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.744+0000 7ff354000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c160050 0x7ff34c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff354000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff34c160430 0x7ff34c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff354000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0ccc70 con 0x7ff34c06d680 2024-09-13T16:52:06.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff354000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0cc060 con 0x7ff34c160050 2024-09-13T16:52:06.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff354000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0cc340 con 0x7ff34c160430 2024-09-13T16:52:06.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c14f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44910/0 (socket says 172.21.15.3:44910) 2024-09-13T16:52:06.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 -- 172.21.15.3:0/1309093962 learned_addr learned my addr 172.21.15.3:0/1309093962 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:06.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 -- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff34c160430 msgr2=0x7ff34c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff34c160430 0x7ff34c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 -- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c160050 msgr2=0x7ff34c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff350c00700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c160050 0x7ff34c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c160050 0x7ff34c1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 -- 172.21.15.3:0/1309093962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344009b30 con 0x7ff34c06d680 2024-09-13T16:52:06.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff350c00700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c160050 0x7ff34c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:06.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff351600700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c14f130 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff34000e9d0 tx=0x7ff34000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff349600700 1 -- 172.21.15.3:0/1309093962 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340009720 con 0x7ff34c06d680 2024-09-13T16:52:06.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c0cc620 con 0x7ff34c06d680 2024-09-13T16:52:06.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff349600700 1 -- 172.21.15.3:0/1309093962 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34000c850 con 0x7ff34c06d680 2024-09-13T16:52:06.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff349600700 1 -- 172.21.15.3:0/1309093962 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34000cbc0 con 0x7ff34c06d680 2024-09-13T16:52:06.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c153d00 con 0x7ff34c06d680 2024-09-13T16:52:06.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.748+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34c06bca0 con 0x7ff34c06d680 2024-09-13T16:52:06.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.752+0000 7ff349600700 1 -- 172.21.15.3:0/1309093962 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff34000cd20 con 0x7ff34c06d680 2024-09-13T16:52:06.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.752+0000 7ff349600700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff32c066730 0x7ff32c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.752+0000 7ff349600700 1 -- 172.21.15.3:0/1309093962 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff340084330 con 0x7ff34c06d680 2024-09-13T16:52:06.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.752+0000 7ff350c00700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff32c066730 0x7ff32c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.756+0000 7ff350c00700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff32c066730 0x7ff32c068bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0cebc0 tx=0x7ff338009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.760+0000 7ff349600700 1 -- 172.21.15.3:0/1309093962 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff3400515f0 con 0x7ff34c06d680 2024-09-13T16:52:06.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.952+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff34c0cf1a0 con 0x7ff34c06d680 2024-09-13T16:52:06.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.956+0000 7ff349600700 1 -- 172.21.15.3:0/1309093962 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16726 (secure 0 0 0) 0x7ff340054c10 con 0x7ff34c06d680 2024-09-13T16:52:06.961 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:06.961 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:04.054168+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:01.009263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:02.059017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:06.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff32c066730 msgr2=0x7ff32c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff32c066730 0x7ff32c068bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0cebc0 tx=0x7ff338009040 comp rx=0 tx=0).stop 2024-09-13T16:52:06.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 msgr2=0x7ff34c14f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c14f130 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff34000e9d0 tx=0x7ff34000ed90 comp rx=0 tx=0).stop 2024-09-13T16:52:06.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 shutdown_connections 2024-09-13T16:52:06.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff32c066730 0x7ff32c068bf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff34c160430 0x7ff34c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34c160050 0x7ff34c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 --2- 172.21.15.3:0/1309093962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34c06d680 0x7ff34c14f130 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 >> 172.21.15.3:0/1309093962 conn(0x7ff34c0707c0 msgr2=0x7ff34c0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:06.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 shutdown_connections 2024-09-13T16:52:06.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:06.960+0000 7ff354000700 1 -- 172.21.15.3:0/1309093962 wait complete. 2024-09-13T16:52:06.988 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T16:52:06.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:52:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.160+0000 7f9611200700 1 Processor -- start 2024-09-13T16:52:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.160+0000 7f9611200700 1 -- start start 2024-09-13T16:52:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.160+0000 7f9611200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f960c0cc060 0x7f960c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.160+0000 7f9611200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f960c06d740 0x7f960c06aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.160+0000 7f9611200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 0x7f960c15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.160+0000 7f9611200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c0cc440 con 0x7f960c06d740 2024-09-13T16:52:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.160+0000 7f9611200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c06a860 con 0x7f960c0cc060 2024-09-13T16:52:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.160+0000 7f9611200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06dbb0 con 0x7f960c06b400 2024-09-13T16:52:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f960c06d740 0x7f960c06aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f960c06d740 0x7f960c06aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44918/0 (socket says 172.21.15.3:44918) 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 -- 172.21.15.3:0/3769275116 learned_addr learned my addr 172.21.15.3:0/3769275116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960b400700 1 --2- 172.21.15.3:0/3769275116 >> v2:172.21.15.157:3300/0 conn(0x7f960c0cc060 0x7f960c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960be00700 1 --2- 172.21.15.3:0/3769275116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 0x7f960c15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 -- 172.21.15.3:0/3769275116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 msgr2=0x7f960c15eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 --2- 172.21.15.3:0/3769275116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 0x7f960c15eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 -- 172.21.15.3:0/3769275116 >> v2:172.21.15.157:3300/0 conn(0x7f960c0cc060 msgr2=0x7f960c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 --2- 172.21.15.3:0/3769275116 >> v2:172.21.15.157:3300/0 conn(0x7f960c0cc060 0x7f960c15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 -- 172.21.15.3:0/3769275116 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c15b5b0 con 0x7f960c06d740 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960aa00700 1 --2- 172.21.15.3:0/3769275116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c06d740 0x7f960c06aec0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009910 tx=0x7f95f8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98997b2ea692dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960a000700 1 -- 172.21.15.3:0/3769275116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8011830 con 0x7f960c06d740 2024-09-13T16:52:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960a000700 1 -- 172.21.15.3:0/3769275116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f8011990 con 0x7f960c06d740 2024-09-13T16:52:07.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f960a000700 1 -- 172.21.15.3:0/3769275116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8011d40 con 0x7f960c06d740 2024-09-13T16:52:07.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 -- 172.21.15.3:0/3769275116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c06d740 msgr2=0x7f960c06aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 --2- 172.21.15.3:0/3769275116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c06d740 0x7f960c06aec0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009910 tx=0x7f95f8009c20 comp rx=0 tx=0).stop 2024-09-13T16:52:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 -- 172.21.15.3:0/3769275116 shutdown_connections 2024-09-13T16:52:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 --2- 172.21.15.3:0/3769275116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c06d740 0x7f960c06aec0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 --2- 172.21.15.3:0/3769275116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 0x7f960c15eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 --2- 172.21.15.3:0/3769275116 >> v2:172.21.15.157:3300/0 conn(0x7f960c0cc060 0x7f960c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 -- 172.21.15.3:0/3769275116 >> 172.21.15.3:0/3769275116 conn(0x7f960c071d40 msgr2=0x7f960c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 -- 172.21.15.3:0/3769275116 shutdown_connections 2024-09-13T16:52:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.164+0000 7f9611200700 1 -- 172.21.15.3:0/3769275116 wait complete. 2024-09-13T16:52:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.168+0000 7f9611200700 1 Processor -- start 2024-09-13T16:52:07.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.168+0000 7f9611200700 1 -- start start 2024-09-13T16:52:07.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.168+0000 7f9611200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 0x7f960c154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.168+0000 7f9611200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06d740 0x7f960c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9611200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c0cc060 0x7f960c15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9611200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c153f70 con 0x7f960c0cc060 2024-09-13T16:52:07.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9611200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c152ed0 con 0x7f960c06d740 2024-09-13T16:52:07.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9611200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c1531b0 con 0x7f960c06b400 2024-09-13T16:52:07.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c0cc060 0x7f960c15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c0cc060 0x7f960c15d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44928/0 (socket says 172.21.15.3:44928) 2024-09-13T16:52:07.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 -- 172.21.15.3:0/2480864809 learned_addr learned my addr 172.21.15.3:0/2480864809 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:07.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960aa00700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06d740 0x7f960c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960b400700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 0x7f960c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 -- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 msgr2=0x7f960c154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 0x7f960c154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 -- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06d740 msgr2=0x7f960c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06d740 0x7f960c15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 -- 172.21.15.3:0/2480864809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f8009570 con 0x7f960c0cc060 2024-09-13T16:52:07.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f960be00700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c0cc060 0x7f960c15d720 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f960400eed0 tx=0x7f960400b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9603400700 1 -- 172.21.15.3:0/2480864809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604010040 con 0x7f960c0cc060 2024-09-13T16:52:07.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9603400700 1 -- 172.21.15.3:0/2480864809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960400bb70 con 0x7f960c0cc060 2024-09-13T16:52:07.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c153490 con 0x7f960c0cc060 2024-09-13T16:52:07.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9603400700 1 -- 172.21.15.3:0/2480864809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604014400 con 0x7f960c0cc060 2024-09-13T16:52:07.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c153a50 con 0x7f960c0cc060 2024-09-13T16:52:07.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.172+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960c06aec0 con 0x7f960c0cc060 2024-09-13T16:52:07.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.176+0000 7f9603400700 1 -- 172.21.15.3:0/2480864809 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9604014560 con 0x7f960c0cc060 2024-09-13T16:52:07.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.176+0000 7f9603400700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95ec066730 0x7f95ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.176+0000 7f9603400700 1 -- 172.21.15.3:0/2480864809 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9604084760 con 0x7f960c0cc060 2024-09-13T16:52:07.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.176+0000 7f960b400700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95ec066730 0x7f95ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.180+0000 7f960b400700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95ec066730 0x7f95ec068bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009b80 tx=0x7f95fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.184+0000 7f9603400700 1 -- 172.21.15.3:0/2480864809 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96040518f0 con 0x7f960c0cc060 2024-09-13T16:52:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.384+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f960c0b5880 con 0x7f95ec066730 2024-09-13T16:52:07.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.400+0000 7f9603400700 1 -- 172.21.15.3:0/2480864809 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287057 (secure 0 0 0) 0x7f960c0b5880 con 0x7f95ec066730 2024-09-13T16:52:07.412 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:07.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95ec066730 msgr2=0x7f95ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95ec066730 0x7f95ec068bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009b80 tx=0x7f95fc008040 comp rx=0 tx=0).stop 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c0cc060 msgr2=0x7f960c15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c0cc060 0x7f960c15d720 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f960400eed0 tx=0x7f960400b0b0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 shutdown_connections 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f95ec066730 0x7f95ec068bf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f960c0cc060 0x7f960c15d720 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06d740 0x7f960c15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 --2- 172.21.15.3:0/2480864809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960c06b400 0x7f960c154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 >> 172.21.15.3:0/2480864809 conn(0x7f960c071d40 msgr2=0x7f960c15e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 shutdown_connections 2024-09-13T16:52:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.408+0000 7f9611200700 1 -- 172.21.15.3:0/2480864809 wait complete. 2024-09-13T16:52:07.416 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:52:07.437 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-13T16:52:07.229588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237328,"kb_used_data":4756,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737392,"statfs":{"total":773094113280,"available":772851089408,"internally_reserved":0,"allocated":4870144,"data_stored":2748560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.674890"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068248+0000","last_change":"2024-09-13T16:52:03.080863+0000","last_active":"2024-09-13T16:52:04.068248+0000","last_peered":"2024-09-13T16:52:04.068248+0000","last_clean":"2024-09-13T16:52:04.068248+0000","last_became_active":"2024-09-13T16:52:03.080642+0000","last_became_peered":"2024-09-13T16:52:03.080642+0000","last_unstale":"2024-09-13T16:52:04.068248+0000","last_undegraded":"2024-09-13T16:52:04.068248+0000","last_fullsized":"2024-09-13T16:52:04.068248+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:36.778123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067632+0000","last_change":"2024-09-13T16:52:03.092688+0000","last_active":"2024-09-13T16:52:04.067632+0000","last_peered":"2024-09-13T16:52:04.067632+0000","last_clean":"2024-09-13T16:52:04.067632+0000","last_became_active":"2024-09-13T16:52:03.092437+0000","last_became_peered":"2024-09-13T16:52:03.092437+0000","last_unstale":"2024-09-13T16:52:04.067632+0000","last_undegraded":"2024-09-13T16:52:04.067632+0000","last_fullsized":"2024-09-13T16:52:04.067632+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:56.542780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.085110+0000","last_change":"2024-09-13T16:52:03.082017+0000","last_active":"2024-09-13T16:52:03.085110+0000","last_peered":"2024-09-13T16:52:03.085110+0000","last_clean":"2024-09-13T16:52:03.085110+0000","last_became_active":"2024-09-13T16:52:03.081747+0000","last_became_peered":"2024-09-13T16:52:03.081747+0000","last_unstale":"2024-09-13T16:52:03.085110+0000","last_undegraded":"2024-09-13T16:52:03.085110+0000","last_fullsized":"2024-09-13T16:52:03.085110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:11.241424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.089846+0000","last_change":"2024-09-13T16:52:03.086629+0000","last_active":"2024-09-13T16:52:03.089846+0000","last_peered":"2024-09-13T16:52:03.089846+0000","last_clean":"2024-09-13T16:52:03.089846+0000","last_became_active":"2024-09-13T16:52:03.086364+0000","last_became_peered":"2024-09-13T16:52:03.086364+0000","last_unstale":"2024-09-13T16:52:03.089846+0000","last_undegraded":"2024-09-13T16:52:03.089846+0000","last_fullsized":"2024-09-13T16:52:03.089846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:55.865168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.091626+0000","last_change":"2024-09-13T16:52:03.085436+0000","last_active":"2024-09-13T16:52:03.091626+0000","last_peered":"2024-09-13T16:52:03.091626+0000","last_clean":"2024-09-13T16:52:03.091626+0000","last_became_active":"2024-09-13T16:52:03.085189+0000","last_became_peered":"2024-09-13T16:52:03.085189+0000","last_unstale":"2024-09-13T16:52:03.091626+0000","last_undegraded":"2024-09-13T16:52:03.091626+0000","last_fullsized":"2024-09-13T16:52:03.091626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:19.293991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.100240+0000","last_change":"2024-09-13T16:52:03.098673+0000","last_active":"2024-09-13T16:52:03.100240+0000","last_peered":"2024-09-13T16:52:03.100240+0000","last_clean":"2024-09-13T16:52:03.100240+0000","last_became_active":"2024-09-13T16:52:03.098431+0000","last_became_peered":"2024-09-13T16:52:03.098431+0000","last_unstale":"2024-09-13T16:52:03.100240+0000","last_undegraded":"2024-09-13T16:52:03.100240+0000","last_fullsized":"2024-09-13T16:52:03.100240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:11.241424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068232+0000","last_change":"2024-09-13T16:52:03.081059+0000","last_active":"2024-09-13T16:52:04.068232+0000","last_peered":"2024-09-13T16:52:04.068232+0000","last_clean":"2024-09-13T16:52:04.068232+0000","last_became_active":"2024-09-13T16:52:03.080623+0000","last_became_peered":"2024-09-13T16:52:03.080623+0000","last_unstale":"2024-09-13T16:52:04.068232+0000","last_undegraded":"2024-09-13T16:52:04.068232+0000","last_fullsized":"2024-09-13T16:52:04.068232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:48.651139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067490+0000","last_change":"2024-09-13T16:52:03.092917+0000","last_active":"2024-09-13T16:52:04.067490+0000","last_peered":"2024-09-13T16:52:04.067490+0000","last_clean":"2024-09-13T16:52:04.067490+0000","last_became_active":"2024-09-13T16:52:03.083916+0000","last_became_peered":"2024-09-13T16:52:03.083916+0000","last_unstale":"2024-09-13T16:52:04.067490+0000","last_undegraded":"2024-09-13T16:52:04.067490+0000","last_fullsized":"2024-09-13T16:52:04.067490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:02.499074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.090179+0000","last_change":"2024-09-13T16:52:03.089092+0000","last_active":"2024-09-13T16:52:03.090179+0000","last_peered":"2024-09-13T16:52:03.090179+0000","last_clean":"2024-09-13T16:52:03.090179+0000","last_became_active":"2024-09-13T16:52:03.088825+0000","last_became_peered":"2024-09-13T16:52:03.088825+0000","last_unstale":"2024-09-13T16:52:03.090179+0000","last_undegraded":"2024-09-13T16:52:03.090179+0000","last_fullsized":"2024-09-13T16:52:03.090179+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:06.994719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068775+0000","last_change":"2024-09-13T16:52:03.072425+0000","last_active":"2024-09-13T16:52:04.068775+0000","last_peered":"2024-09-13T16:52:04.068775+0000","last_clean":"2024-09-13T16:52:04.068775+0000","last_became_active":"2024-09-13T16:52:03.072102+0000","last_became_peered":"2024-09-13T16:52:03.072102+0000","last_unstale":"2024-09-13T16:52:04.068775+0000","last_undegraded":"2024-09-13T16:52:04.068775+0000","last_fullsized":"2024-09-13T16:52:04.068775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:06.544382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069267+0000","last_change":"2024-09-13T16:52:03.086328+0000","last_active":"2024-09-13T16:52:04.069267+0000","last_peered":"2024-09-13T16:52:04.069267+0000","last_clean":"2024-09-13T16:52:04.069267+0000","last_became_active":"2024-09-13T16:52:03.086079+0000","last_became_peered":"2024-09-13T16:52:03.086079+0000","last_unstale":"2024-09-13T16:52:04.069267+0000","last_undegraded":"2024-09-13T16:52:04.069267+0000","last_fullsized":"2024-09-13T16:52:04.069267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:46.850446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.083253+0000","last_change":"2024-09-13T16:52:03.078958+0000","last_active":"2024-09-13T16:52:03.083253+0000","last_peered":"2024-09-13T16:52:03.083253+0000","last_clean":"2024-09-13T16:52:03.083253+0000","last_became_active":"2024-09-13T16:52:03.078698+0000","last_became_peered":"2024-09-13T16:52:03.078698+0000","last_unstale":"2024-09-13T16:52:03.083253+0000","last_undegraded":"2024-09-13T16:52:03.083253+0000","last_fullsized":"2024-09-13T16:52:03.083253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:40.831264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068478+0000","last_change":"2024-09-13T16:52:03.078315+0000","last_active":"2024-09-13T16:52:04.068478+0000","last_peered":"2024-09-13T16:52:04.068478+0000","last_clean":"2024-09-13T16:52:04.068478+0000","last_became_active":"2024-09-13T16:52:03.074000+0000","last_became_peered":"2024-09-13T16:52:03.074000+0000","last_unstale":"2024-09-13T16:52:04.068478+0000","last_undegraded":"2024-09-13T16:52:04.068478+0000","last_fullsized":"2024-09-13T16:52:04.068478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:48.611064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069532+0000","last_change":"2024-09-13T16:52:03.090755+0000","last_active":"2024-09-13T16:52:04.069532+0000","last_peered":"2024-09-13T16:52:04.069532+0000","last_clean":"2024-09-13T16:52:04.069532+0000","last_became_active":"2024-09-13T16:52:03.090511+0000","last_became_peered":"2024-09-13T16:52:03.090511+0000","last_unstale":"2024-09-13T16:52:04.069532+0000","last_undegraded":"2024-09-13T16:52:04.069532+0000","last_fullsized":"2024-09-13T16:52:04.069532+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:11.241424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068450+0000","last_change":"2024-09-13T16:52:03.080866+0000","last_active":"2024-09-13T16:52:04.068450+0000","last_peered":"2024-09-13T16:52:04.068450+0000","last_clean":"2024-09-13T16:52:04.068450+0000","last_became_active":"2024-09-13T16:52:03.080264+0000","last_became_peered":"2024-09-13T16:52:03.080264+0000","last_unstale":"2024-09-13T16:52:04.068450+0000","last_undegraded":"2024-09-13T16:52:04.068450+0000","last_fullsized":"2024-09-13T16:52:04.068450+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:09.534736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.094881+0000","last_change":"2024-09-13T16:52:03.088360+0000","last_active":"2024-09-13T16:52:03.094881+0000","last_peered":"2024-09-13T16:52:03.094881+0000","last_clean":"2024-09-13T16:52:03.094881+0000","last_became_active":"2024-09-13T16:52:03.084442+0000","last_became_peered":"2024-09-13T16:52:03.084442+0000","last_unstale":"2024-09-13T16:52:03.094881+0000","last_undegraded":"2024-09-13T16:52:03.094881+0000","last_fullsized":"2024-09-13T16:52:03.094881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:18.226039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.095941+0000","last_change":"2024-09-13T16:52:03.093441+0000","last_active":"2024-09-13T16:52:03.095941+0000","last_peered":"2024-09-13T16:52:03.095941+0000","last_clean":"2024-09-13T16:52:03.095941+0000","last_became_active":"2024-09-13T16:52:03.093113+0000","last_became_peered":"2024-09-13T16:52:03.093113+0000","last_unstale":"2024-09-13T16:52:03.095941+0000","last_undegraded":"2024-09-13T16:52:03.095941+0000","last_fullsized":"2024-09-13T16:52:03.095941+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:14.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.081864+0000","last_change":"2024-09-13T16:52:03.076797+0000","last_active":"2024-09-13T16:52:03.081864+0000","last_peered":"2024-09-13T16:52:03.081864+0000","last_clean":"2024-09-13T16:52:03.081864+0000","last_became_active":"2024-09-13T16:52:03.076495+0000","last_became_peered":"2024-09-13T16:52:03.076495+0000","last_unstale":"2024-09-13T16:52:03.081864+0000","last_undegraded":"2024-09-13T16:52:03.081864+0000","last_fullsized":"2024-09-13T16:52:03.081864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:47.136150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.090490+0000","last_change":"2024-09-13T16:52:03.085759+0000","last_active":"2024-09-13T16:52:03.090490+0000","last_peered":"2024-09-13T16:52:03.090490+0000","last_clean":"2024-09-13T16:52:03.090490+0000","last_became_active":"2024-09-13T16:52:03.085511+0000","last_became_peered":"2024-09-13T16:52:03.085511+0000","last_unstale":"2024-09-13T16:52:03.090490+0000","last_undegraded":"2024-09-13T16:52:03.090490+0000","last_fullsized":"2024-09-13T16:52:03.090490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:41.160361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.096134+0000","last_change":"2024-09-13T16:52:03.091312+0000","last_active":"2024-09-13T16:52:03.096134+0000","last_peered":"2024-09-13T16:52:03.096134+0000","last_clean":"2024-09-13T16:52:03.096134+0000","last_became_active":"2024-09-13T16:52:03.091043+0000","last_became_peered":"2024-09-13T16:52:03.091043+0000","last_unstale":"2024-09-13T16:52:03.096134+0000","last_undegraded":"2024-09-13T16:52:03.096134+0000","last_fullsized":"2024-09-13T16:52:03.096134+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:29.500537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068232+0000","last_change":"2024-09-13T16:52:03.085790+0000","last_active":"2024-09-13T16:52:04.068232+0000","last_peered":"2024-09-13T16:52:04.068232+0000","last_clean":"2024-09-13T16:52:04.068232+0000","last_became_active":"2024-09-13T16:52:03.083654+0000","last_became_peered":"2024-09-13T16:52:03.083654+0000","last_unstale":"2024-09-13T16:52:04.068232+0000","last_undegraded":"2024-09-13T16:52:04.068232+0000","last_fullsized":"2024-09-13T16:52:04.068232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:09.761884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069986+0000","last_change":"2024-09-13T16:52:03.095433+0000","last_active":"2024-09-13T16:52:04.069986+0000","last_peered":"2024-09-13T16:52:04.069986+0000","last_clean":"2024-09-13T16:52:04.069986+0000","last_became_active":"2024-09-13T16:52:03.095122+0000","last_became_peered":"2024-09-13T16:52:03.095122+0000","last_unstale":"2024-09-13T16:52:04.069986+0000","last_undegraded":"2024-09-13T16:52:04.069986+0000","last_fullsized":"2024-09-13T16:52:04.069986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:10.717561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.092998+0000","last_change":"2024-09-13T16:52:03.082641+0000","last_active":"2024-09-13T16:52:03.092998+0000","last_peered":"2024-09-13T16:52:03.092998+0000","last_clean":"2024-09-13T16:52:03.092998+0000","last_became_active":"2024-09-13T16:52:03.082166+0000","last_became_peered":"2024-09-13T16:52:03.082166+0000","last_unstale":"2024-09-13T16:52:03.092998+0000","last_undegraded":"2024-09-13T16:52:03.092998+0000","last_fullsized":"2024-09-13T16:52:03.092998+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:33.230927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.077021+0000","last_change":"2024-09-13T16:52:03.071454+0000","last_active":"2024-09-13T16:52:03.077021+0000","last_peered":"2024-09-13T16:52:03.077021+0000","last_clean":"2024-09-13T16:52:03.077021+0000","last_became_active":"2024-09-13T16:52:03.071161+0000","last_became_peered":"2024-09-13T16:52:03.071161+0000","last_unstale":"2024-09-13T16:52:03.077021+0000","last_undegraded":"2024-09-13T16:52:03.077021+0000","last_fullsized":"2024-09-13T16:52:03.077021+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:29.699627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.093641+0000","last_change":"2024-09-13T16:52:03.090404+0000","last_active":"2024-09-13T16:52:03.093641+0000","last_peered":"2024-09-13T16:52:03.093641+0000","last_clean":"2024-09-13T16:52:03.093641+0000","last_became_active":"2024-09-13T16:52:03.082870+0000","last_became_peered":"2024-09-13T16:52:03.082870+0000","last_unstale":"2024-09-13T16:52:03.093641+0000","last_undegraded":"2024-09-13T16:52:03.093641+0000","last_fullsized":"2024-09-13T16:52:03.093641+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:40.831264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.090580+0000","last_change":"2024-09-13T16:52:03.080280+0000","last_active":"2024-09-13T16:52:03.090580+0000","last_peered":"2024-09-13T16:52:03.090580+0000","last_clean":"2024-09-13T16:52:03.090580+0000","last_became_active":"2024-09-13T16:52:03.080020+0000","last_became_peered":"2024-09-13T16:52:03.080020+0000","last_unstale":"2024-09-13T16:52:03.090580+0000","last_undegraded":"2024-09-13T16:52:03.090580+0000","last_fullsized":"2024-09-13T16:52:03.090580+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:27:40.756415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.082430+0000","last_change":"2024-09-13T16:52:03.077877+0000","last_active":"2024-09-13T16:52:03.082430+0000","last_peered":"2024-09-13T16:52:03.082430+0000","last_clean":"2024-09-13T16:52:03.082430+0000","last_became_active":"2024-09-13T16:52:03.077637+0000","last_became_peered":"2024-09-13T16:52:03.077637+0000","last_unstale":"2024-09-13T16:52:03.082430+0000","last_undegraded":"2024-09-13T16:52:03.082430+0000","last_fullsized":"2024-09-13T16:52:03.082430+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:18.226039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068773+0000","last_change":"2024-09-13T16:52:03.073636+0000","last_active":"2024-09-13T16:52:04.068773+0000","last_peered":"2024-09-13T16:52:04.068773+0000","last_clean":"2024-09-13T16:52:04.068773+0000","last_became_active":"2024-09-13T16:52:03.073012+0000","last_became_peered":"2024-09-13T16:52:03.073012+0000","last_unstale":"2024-09-13T16:52:04.068773+0000","last_undegraded":"2024-09-13T16:52:04.068773+0000","last_fullsized":"2024-09-13T16:52:04.068773+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:41.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.081718+0000","last_change":"2024-09-13T16:52:03.075069+0000","last_active":"2024-09-13T16:52:03.081718+0000","last_peered":"2024-09-13T16:52:03.081718+0000","last_clean":"2024-09-13T16:52:03.081718+0000","last_became_active":"2024-09-13T16:52:03.074351+0000","last_became_peered":"2024-09-13T16:52:03.074351+0000","last_unstale":"2024-09-13T16:52:03.081718+0000","last_undegraded":"2024-09-13T16:52:03.081718+0000","last_fullsized":"2024-09-13T16:52:03.081718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:14.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.082038+0000","last_change":"2024-09-13T16:52:03.077430+0000","last_active":"2024-09-13T16:52:03.082038+0000","last_peered":"2024-09-13T16:52:03.082038+0000","last_clean":"2024-09-13T16:52:03.082038+0000","last_became_active":"2024-09-13T16:52:03.077177+0000","last_became_peered":"2024-09-13T16:52:03.077177+0000","last_unstale":"2024-09-13T16:52:03.082038+0000","last_undegraded":"2024-09-13T16:52:03.082038+0000","last_fullsized":"2024-09-13T16:52:03.082038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:05.911241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.093017+0000","last_change":"2024-09-13T16:52:03.083046+0000","last_active":"2024-09-13T16:52:03.093017+0000","last_peered":"2024-09-13T16:52:03.093017+0000","last_clean":"2024-09-13T16:52:03.093017+0000","last_became_active":"2024-09-13T16:52:03.082804+0000","last_became_peered":"2024-09-13T16:52:03.082804+0000","last_unstale":"2024-09-13T16:52:03.093017+0000","last_undegraded":"2024-09-13T16:52:03.093017+0000","last_fullsized":"2024-09-13T16:52:03.093017+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:53.962488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.086787+0000","last_change":"2024-09-13T16:52:03.077907+0000","last_active":"2024-09-13T16:52:03.086787+0000","last_peered":"2024-09-13T16:52:03.086787+0000","last_clean":"2024-09-13T16:52:03.086787+0000","last_became_active":"2024-09-13T16:52:03.077631+0000","last_became_peered":"2024-09-13T16:52:03.077631+0000","last_unstale":"2024-09-13T16:52:03.086787+0000","last_undegraded":"2024-09-13T16:52:03.086787+0000","last_fullsized":"2024-09-13T16:52:03.086787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:58.383411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.094495+0000","last_change":"2024-09-13T16:52:03.087742+0000","last_active":"2024-09-13T16:52:03.094495+0000","last_peered":"2024-09-13T16:52:03.094495+0000","last_clean":"2024-09-13T16:52:03.094495+0000","last_became_active":"2024-09-13T16:52:03.087514+0000","last_became_peered":"2024-09-13T16:52:03.087514+0000","last_unstale":"2024-09-13T16:52:03.094495+0000","last_undegraded":"2024-09-13T16:52:03.094495+0000","last_fullsized":"2024-09-13T16:52:03.094495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:05.911241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.097597+0000","last_change":"2024-09-13T16:52:02.585632+0000","last_active":"2024-09-13T16:52:03.097597+0000","last_peered":"2024-09-13T16:52:03.097597+0000","last_clean":"2024-09-13T16:52:03.097597+0000","last_became_active":"2024-09-13T16:52:02.585369+0000","last_became_peered":"2024-09-13T16:52:02.585369+0000","last_unstale":"2024-09-13T16:52:03.097597+0000","last_undegraded":"2024-09-13T16:52:03.097597+0000","last_fullsized":"2024-09-13T16:52:03.097597+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:14.240207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.090881+0000","last_change":"2024-09-13T16:52:03.084507+0000","last_active":"2024-09-13T16:52:03.090881+0000","last_peered":"2024-09-13T16:52:03.090881+0000","last_clean":"2024-09-13T16:52:03.090881+0000","last_became_active":"2024-09-13T16:52:03.084213+0000","last_became_peered":"2024-09-13T16:52:03.084213+0000","last_unstale":"2024-09-13T16:52:03.090881+0000","last_undegraded":"2024-09-13T16:52:03.090881+0000","last_fullsized":"2024-09-13T16:52:03.090881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:19.665369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.095015+0000","last_change":"2024-09-13T16:52:02.583744+0000","last_active":"2024-09-13T16:52:03.095015+0000","last_peered":"2024-09-13T16:52:03.095015+0000","last_clean":"2024-09-13T16:52:03.095015+0000","last_became_active":"2024-09-13T16:52:02.583472+0000","last_became_peered":"2024-09-13T16:52:02.583472+0000","last_unstale":"2024-09-13T16:52:03.095015+0000","last_undegraded":"2024-09-13T16:52:03.095015+0000","last_fullsized":"2024-09-13T16:52:03.095015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:40.149686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069994+0000","last_change":"2024-09-13T16:52:03.088688+0000","last_active":"2024-09-13T16:52:04.069994+0000","last_peered":"2024-09-13T16:52:04.069994+0000","last_clean":"2024-09-13T16:52:04.069994+0000","last_became_active":"2024-09-13T16:52:03.088276+0000","last_became_peered":"2024-09-13T16:52:03.088276+0000","last_unstale":"2024-09-13T16:52:04.069994+0000","last_undegraded":"2024-09-13T16:52:04.069994+0000","last_fullsized":"2024-09-13T16:52:04.069994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:27:40.756415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.099194+0000","last_change":"2024-09-13T16:52:02.587230+0000","last_active":"2024-09-13T16:52:03.099194+0000","last_peered":"2024-09-13T16:52:03.099194+0000","last_clean":"2024-09-13T16:52:03.099194+0000","last_became_active":"2024-09-13T16:52:02.586966+0000","last_became_peered":"2024-09-13T16:52:02.586966+0000","last_unstale":"2024-09-13T16:52:03.099194+0000","last_undegraded":"2024-09-13T16:52:03.099194+0000","last_fullsized":"2024-09-13T16:52:03.099194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:40.149686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068196+0000","last_change":"2024-09-13T16:52:03.100886+0000","last_active":"2024-09-13T16:52:04.068196+0000","last_peered":"2024-09-13T16:52:04.068196+0000","last_clean":"2024-09-13T16:52:04.068196+0000","last_became_active":"2024-09-13T16:52:03.097629+0000","last_became_peered":"2024-09-13T16:52:03.097629+0000","last_unstale":"2024-09-13T16:52:04.068196+0000","last_undegraded":"2024-09-13T16:52:04.068196+0000","last_fullsized":"2024-09-13T16:52:04.068196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:14.150372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069819+0000","last_change":"2024-09-13T16:52:02.591784+0000","last_active":"2024-09-13T16:52:04.069819+0000","last_peered":"2024-09-13T16:52:04.069819+0000","last_clean":"2024-09-13T16:52:04.069819+0000","last_became_active":"2024-09-13T16:52:02.591544+0000","last_became_peered":"2024-09-13T16:52:02.591544+0000","last_unstale":"2024-09-13T16:52:04.069819+0000","last_undegraded":"2024-09-13T16:52:04.069819+0000","last_fullsized":"2024-09-13T16:52:04.069819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:45.332315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.089747+0000","last_change":"2024-09-13T16:52:03.087733+0000","last_active":"2024-09-13T16:52:03.089747+0000","last_peered":"2024-09-13T16:52:03.089747+0000","last_clean":"2024-09-13T16:52:03.089747+0000","last_became_active":"2024-09-13T16:52:03.087269+0000","last_became_peered":"2024-09-13T16:52:03.087269+0000","last_unstale":"2024-09-13T16:52:03.089747+0000","last_undegraded":"2024-09-13T16:52:03.089747+0000","last_fullsized":"2024-09-13T16:52:03.089747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:19.665369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068350+0000","last_change":"2024-09-13T16:52:02.596848+0000","last_active":"2024-09-13T16:52:04.068350+0000","last_peered":"2024-09-13T16:52:04.068350+0000","last_clean":"2024-09-13T16:52:04.068350+0000","last_became_active":"2024-09-13T16:52:02.596572+0000","last_became_peered":"2024-09-13T16:52:02.596572+0000","last_unstale":"2024-09-13T16:52:04.068350+0000","last_undegraded":"2024-09-13T16:52:04.068350+0000","last_fullsized":"2024-09-13T16:52:04.068350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:31.664763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068479+0000","last_change":"2024-09-13T16:52:03.074248+0000","last_active":"2024-09-13T16:52:04.068479+0000","last_peered":"2024-09-13T16:52:04.068479+0000","last_clean":"2024-09-13T16:52:04.068479+0000","last_became_active":"2024-09-13T16:52:03.073922+0000","last_became_peered":"2024-09-13T16:52:03.073922+0000","last_unstale":"2024-09-13T16:52:04.068479+0000","last_undegraded":"2024-09-13T16:52:04.068479+0000","last_fullsized":"2024-09-13T16:52:04.068479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:06.994719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.095741+0000","last_change":"2024-09-13T16:52:02.584720+0000","last_active":"2024-09-13T16:52:03.095741+0000","last_peered":"2024-09-13T16:52:03.095741+0000","last_clean":"2024-09-13T16:52:03.095741+0000","last_became_active":"2024-09-13T16:52:02.584483+0000","last_became_peered":"2024-09-13T16:52:02.584483+0000","last_unstale":"2024-09-13T16:52:03.095741+0000","last_undegraded":"2024-09-13T16:52:03.095741+0000","last_fullsized":"2024-09-13T16:52:03.095741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:53.118849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069790+0000","last_change":"2024-09-13T16:52:03.084520+0000","last_active":"2024-09-13T16:52:04.069790+0000","last_peered":"2024-09-13T16:52:04.069790+0000","last_clean":"2024-09-13T16:52:04.069790+0000","last_became_active":"2024-09-13T16:52:03.084128+0000","last_became_peered":"2024-09-13T16:52:03.084128+0000","last_unstale":"2024-09-13T16:52:04.069790+0000","last_undegraded":"2024-09-13T16:52:04.069790+0000","last_fullsized":"2024-09-13T16:52:04.069790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:48.651139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.074879+0000","last_change":"2024-09-13T16:52:02.585288+0000","last_active":"2024-09-13T16:52:03.074879+0000","last_peered":"2024-09-13T16:52:03.074879+0000","last_clean":"2024-09-13T16:52:03.074879+0000","last_became_active":"2024-09-13T16:52:02.583943+0000","last_became_peered":"2024-09-13T16:52:02.583943+0000","last_unstale":"2024-09-13T16:52:03.074879+0000","last_undegraded":"2024-09-13T16:52:03.074879+0000","last_fullsized":"2024-09-13T16:52:03.074879+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:45.332315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067662+0000","last_change":"2024-09-13T16:52:03.082320+0000","last_active":"2024-09-13T16:52:04.067662+0000","last_peered":"2024-09-13T16:52:04.067662+0000","last_clean":"2024-09-13T16:52:04.067662+0000","last_became_active":"2024-09-13T16:52:03.080798+0000","last_became_peered":"2024-09-13T16:52:03.080798+0000","last_unstale":"2024-09-13T16:52:04.067662+0000","last_undegraded":"2024-09-13T16:52:04.067662+0000","last_fullsized":"2024-09-13T16:52:04.067662+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:30.022521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069333+0000","last_change":"2024-09-13T16:52:02.586484+0000","last_active":"2024-09-13T16:52:04.069333+0000","last_peered":"2024-09-13T16:52:04.069333+0000","last_clean":"2024-09-13T16:52:04.069333+0000","last_became_active":"2024-09-13T16:52:02.585960+0000","last_became_peered":"2024-09-13T16:52:02.585960+0000","last_unstale":"2024-09-13T16:52:04.069333+0000","last_undegraded":"2024-09-13T16:52:04.069333+0000","last_fullsized":"2024-09-13T16:52:04.069333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:32.220252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.089864+0000","last_change":"2024-09-13T16:52:03.086626+0000","last_active":"2024-09-13T16:52:03.089864+0000","last_peered":"2024-09-13T16:52:03.089864+0000","last_clean":"2024-09-13T16:52:03.089864+0000","last_became_active":"2024-09-13T16:52:03.086309+0000","last_became_peered":"2024-09-13T16:52:03.086309+0000","last_unstale":"2024-09-13T16:52:03.089864+0000","last_undegraded":"2024-09-13T16:52:03.089864+0000","last_fullsized":"2024-09-13T16:52:03.089864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.630647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068280+0000","last_change":"2024-09-13T16:52:02.580190+0000","last_active":"2024-09-13T16:52:04.068280+0000","last_peered":"2024-09-13T16:52:04.068280+0000","last_clean":"2024-09-13T16:52:04.068280+0000","last_became_active":"2024-09-13T16:52:02.579883+0000","last_became_peered":"2024-09-13T16:52:02.579883+0000","last_unstale":"2024-09-13T16:52:04.068280+0000","last_undegraded":"2024-09-13T16:52:04.068280+0000","last_fullsized":"2024-09-13T16:52:04.068280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:09.706886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.094770+0000","last_change":"2024-09-13T16:52:03.088185+0000","last_active":"2024-09-13T16:52:03.094770+0000","last_peered":"2024-09-13T16:52:03.094770+0000","last_clean":"2024-09-13T16:52:03.094770+0000","last_became_active":"2024-09-13T16:52:03.087013+0000","last_became_peered":"2024-09-13T16:52:03.087013+0000","last_unstale":"2024-09-13T16:52:03.094770+0000","last_undegraded":"2024-09-13T16:52:03.094770+0000","last_fullsized":"2024-09-13T16:52:03.094770+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:02.676020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069179+0000","last_change":"2024-09-13T16:52:02.598388+0000","last_active":"2024-09-13T16:52:04.069179+0000","last_peered":"2024-09-13T16:52:04.069179+0000","last_clean":"2024-09-13T16:52:04.069179+0000","last_became_active":"2024-09-13T16:52:02.598113+0000","last_became_peered":"2024-09-13T16:52:02.598113+0000","last_unstale":"2024-09-13T16:52:04.069179+0000","last_undegraded":"2024-09-13T16:52:04.069179+0000","last_fullsized":"2024-09-13T16:52:04.069179+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:52.951813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.085267+0000","last_change":"2024-09-13T16:52:03.083366+0000","last_active":"2024-09-13T16:52:03.085267+0000","last_peered":"2024-09-13T16:52:03.085267+0000","last_clean":"2024-09-13T16:52:03.085267+0000","last_became_active":"2024-09-13T16:52:03.083118+0000","last_became_peered":"2024-09-13T16:52:03.083118+0000","last_unstale":"2024-09-13T16:52:03.085267+0000","last_undegraded":"2024-09-13T16:52:03.085267+0000","last_fullsized":"2024-09-13T16:52:03.085267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:29.500537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069164+0000","last_change":"2024-09-13T16:52:02.589809+0000","last_active":"2024-09-13T16:52:04.069164+0000","last_peered":"2024-09-13T16:52:04.069164+0000","last_clean":"2024-09-13T16:52:04.069164+0000","last_became_active":"2024-09-13T16:52:02.589465+0000","last_became_peered":"2024-09-13T16:52:02.589465+0000","last_unstale":"2024-09-13T16:52:04.069164+0000","last_undegraded":"2024-09-13T16:52:04.069164+0000","last_fullsized":"2024-09-13T16:52:04.069164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:45.332315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069939+0000","last_change":"2024-09-13T16:52:03.085819+0000","last_active":"2024-09-13T16:52:04.069939+0000","last_peered":"2024-09-13T16:52:04.069939+0000","last_clean":"2024-09-13T16:52:04.069939+0000","last_became_active":"2024-09-13T16:52:03.085565+0000","last_became_peered":"2024-09-13T16:52:03.085565+0000","last_unstale":"2024-09-13T16:52:04.069939+0000","last_undegraded":"2024-09-13T16:52:04.069939+0000","last_fullsized":"2024-09-13T16:52:04.069939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:09.534736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068004+0000","last_change":"2024-09-13T16:52:02.596407+0000","last_active":"2024-09-13T16:52:04.068004+0000","last_peered":"2024-09-13T16:52:04.068004+0000","last_clean":"2024-09-13T16:52:04.068004+0000","last_became_active":"2024-09-13T16:52:02.596145+0000","last_became_peered":"2024-09-13T16:52:02.596145+0000","last_unstale":"2024-09-13T16:52:04.068004+0000","last_undegraded":"2024-09-13T16:52:04.068004+0000","last_fullsized":"2024-09-13T16:52:04.068004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:05.984044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.091887+0000","last_change":"2024-09-13T16:52:03.090270+0000","last_active":"2024-09-13T16:52:03.091887+0000","last_peered":"2024-09-13T16:52:03.091887+0000","last_clean":"2024-09-13T16:52:03.091887+0000","last_became_active":"2024-09-13T16:52:03.090030+0000","last_became_peered":"2024-09-13T16:52:03.090030+0000","last_unstale":"2024-09-13T16:52:03.091887+0000","last_undegraded":"2024-09-13T16:52:03.091887+0000","last_fullsized":"2024-09-13T16:52:03.091887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:58.383411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067825+0000","last_change":"2024-09-13T16:52:02.592615+0000","last_active":"2024-09-13T16:52:04.067825+0000","last_peered":"2024-09-13T16:52:04.067825+0000","last_clean":"2024-09-13T16:52:04.067825+0000","last_became_active":"2024-09-13T16:52:02.592297+0000","last_became_peered":"2024-09-13T16:52:02.592297+0000","last_unstale":"2024-09-13T16:52:04.067825+0000","last_undegraded":"2024-09-13T16:52:04.067825+0000","last_fullsized":"2024-09-13T16:52:04.067825+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:10.230749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067584+0000","last_change":"2024-09-13T16:52:03.075624+0000","last_active":"2024-09-13T16:52:04.067584+0000","last_peered":"2024-09-13T16:52:04.067584+0000","last_clean":"2024-09-13T16:52:04.067584+0000","last_became_active":"2024-09-13T16:52:03.075350+0000","last_became_peered":"2024-09-13T16:52:03.075350+0000","last_unstale":"2024-09-13T16:52:04.067584+0000","last_undegraded":"2024-09-13T16:52:04.067584+0000","last_fullsized":"2024-09-13T16:52:04.067584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:52.988622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.074553+0000","last_change":"2024-09-13T16:52:02.595054+0000","last_active":"2024-09-13T16:52:03.074553+0000","last_peered":"2024-09-13T16:52:03.074553+0000","last_clean":"2024-09-13T16:52:03.074553+0000","last_became_active":"2024-09-13T16:52:02.594705+0000","last_became_peered":"2024-09-13T16:52:02.594705+0000","last_unstale":"2024-09-13T16:52:03.074553+0000","last_undegraded":"2024-09-13T16:52:03.074553+0000","last_fullsized":"2024-09-13T16:52:03.074553+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:06.193182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.091983+0000","last_change":"2024-09-13T16:52:03.087279+0000","last_active":"2024-09-13T16:52:03.091983+0000","last_peered":"2024-09-13T16:52:03.091983+0000","last_clean":"2024-09-13T16:52:03.091983+0000","last_became_active":"2024-09-13T16:52:03.086965+0000","last_became_peered":"2024-09-13T16:52:03.086965+0000","last_unstale":"2024-09-13T16:52:03.091983+0000","last_undegraded":"2024-09-13T16:52:03.091983+0000","last_fullsized":"2024-09-13T16:52:03.091983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:15.250882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.062063+0000","last_change":"2024-09-13T16:52:02.584595+0000","last_active":"2024-09-13T16:52:03.062063+0000","last_peered":"2024-09-13T16:52:03.062063+0000","last_clean":"2024-09-13T16:52:03.062063+0000","last_became_active":"2024-09-13T16:52:02.584354+0000","last_became_peered":"2024-09-13T16:52:02.584354+0000","last_unstale":"2024-09-13T16:52:03.062063+0000","last_undegraded":"2024-09-13T16:52:03.062063+0000","last_fullsized":"2024-09-13T16:52:03.062063+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:18.283316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067613+0000","last_change":"2024-09-13T16:52:03.092913+0000","last_active":"2024-09-13T16:52:04.067613+0000","last_peered":"2024-09-13T16:52:04.067613+0000","last_clean":"2024-09-13T16:52:04.067613+0000","last_became_active":"2024-09-13T16:52:03.092656+0000","last_became_peered":"2024-09-13T16:52:03.092656+0000","last_unstale":"2024-09-13T16:52:04.067613+0000","last_undegraded":"2024-09-13T16:52:04.067613+0000","last_fullsized":"2024-09-13T16:52:04.067613+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:02:26.664729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.096718+0000","last_change":"2024-09-13T16:52:02.584744+0000","last_active":"2024-09-13T16:52:03.096718+0000","last_peered":"2024-09-13T16:52:03.096718+0000","last_clean":"2024-09-13T16:52:03.096718+0000","last_became_active":"2024-09-13T16:52:02.584454+0000","last_became_peered":"2024-09-13T16:52:02.584454+0000","last_unstale":"2024-09-13T16:52:03.096718+0000","last_undegraded":"2024-09-13T16:52:03.096718+0000","last_fullsized":"2024-09-13T16:52:03.096718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:30.321115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067896+0000","last_change":"2024-09-13T16:52:03.082946+0000","last_active":"2024-09-13T16:52:04.067896+0000","last_peered":"2024-09-13T16:52:04.067896+0000","last_clean":"2024-09-13T16:52:04.067896+0000","last_became_active":"2024-09-13T16:52:03.082687+0000","last_became_peered":"2024-09-13T16:52:03.082687+0000","last_unstale":"2024-09-13T16:52:04.067896+0000","last_undegraded":"2024-09-13T16:52:04.067896+0000","last_fullsized":"2024-09-13T16:52:04.067896+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:02.621626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069057+0000","last_change":"2024-09-13T16:52:02.589160+0000","last_active":"2024-09-13T16:52:04.069057+0000","last_peered":"2024-09-13T16:52:04.069057+0000","last_clean":"2024-09-13T16:52:04.069057+0000","last_became_active":"2024-09-13T16:52:02.588178+0000","last_became_peered":"2024-09-13T16:52:02.588178+0000","last_unstale":"2024-09-13T16:52:04.069057+0000","last_undegraded":"2024-09-13T16:52:04.069057+0000","last_fullsized":"2024-09-13T16:52:04.069057+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:32.220252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068973+0000","last_change":"2024-09-13T16:52:03.080651+0000","last_active":"2024-09-13T16:52:04.068973+0000","last_peered":"2024-09-13T16:52:04.068973+0000","last_clean":"2024-09-13T16:52:04.068973+0000","last_became_active":"2024-09-13T16:52:03.080071+0000","last_became_peered":"2024-09-13T16:52:03.080071+0000","last_unstale":"2024-09-13T16:52:04.068973+0000","last_undegraded":"2024-09-13T16:52:04.068973+0000","last_fullsized":"2024-09-13T16:52:04.068973+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:44.839062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.070031+0000","last_change":"2024-09-13T16:52:02.593248+0000","last_active":"2024-09-13T16:52:04.070031+0000","last_peered":"2024-09-13T16:52:04.070031+0000","last_clean":"2024-09-13T16:52:04.070031+0000","last_became_active":"2024-09-13T16:52:02.592722+0000","last_became_peered":"2024-09-13T16:52:02.592722+0000","last_unstale":"2024-09-13T16:52:04.070031+0000","last_undegraded":"2024-09-13T16:52:04.070031+0000","last_fullsized":"2024-09-13T16:52:04.070031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:52.951813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069811+0000","last_change":"2024-09-13T16:52:03.095454+0000","last_active":"2024-09-13T16:52:04.069811+0000","last_peered":"2024-09-13T16:52:04.069811+0000","last_clean":"2024-09-13T16:52:04.069811+0000","last_became_active":"2024-09-13T16:52:03.095191+0000","last_became_peered":"2024-09-13T16:52:03.095191+0000","last_unstale":"2024-09-13T16:52:04.069811+0000","last_undegraded":"2024-09-13T16:52:04.069811+0000","last_fullsized":"2024-09-13T16:52:04.069811+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:11:21.464886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.085295+0000","last_change":"2024-09-13T16:52:02.585970+0000","last_active":"2024-09-13T16:52:03.085295+0000","last_peered":"2024-09-13T16:52:03.085295+0000","last_clean":"2024-09-13T16:52:03.085295+0000","last_became_active":"2024-09-13T16:52:02.585718+0000","last_became_peered":"2024-09-13T16:52:02.585718+0000","last_unstale":"2024-09-13T16:52:03.085295+0000","last_undegraded":"2024-09-13T16:52:03.085295+0000","last_fullsized":"2024-09-13T16:52:03.085295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:09:01.467945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.099687+0000","last_change":"2024-09-13T16:52:03.098358+0000","last_active":"2024-09-13T16:52:03.099687+0000","last_peered":"2024-09-13T16:52:03.099687+0000","last_clean":"2024-09-13T16:52:03.099687+0000","last_became_active":"2024-09-13T16:52:03.097914+0000","last_became_peered":"2024-09-13T16:52:03.097914+0000","last_unstale":"2024-09-13T16:52:03.099687+0000","last_undegraded":"2024-09-13T16:52:03.099687+0000","last_fullsized":"2024-09-13T16:52:03.099687+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:46.496049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.075011+0000","last_change":"2024-09-13T16:52:02.587282+0000","last_active":"2024-09-13T16:52:03.075011+0000","last_peered":"2024-09-13T16:52:03.075011+0000","last_clean":"2024-09-13T16:52:03.075011+0000","last_became_active":"2024-09-13T16:52:02.587019+0000","last_became_peered":"2024-09-13T16:52:02.587019+0000","last_unstale":"2024-09-13T16:52:03.075011+0000","last_undegraded":"2024-09-13T16:52:03.075011+0000","last_fullsized":"2024-09-13T16:52:03.075011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:32.220252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.081093+0000","last_change":"2024-09-13T16:52:03.079064+0000","last_active":"2024-09-13T16:52:03.081093+0000","last_peered":"2024-09-13T16:52:03.081093+0000","last_clean":"2024-09-13T16:52:03.081093+0000","last_became_active":"2024-09-13T16:52:03.078804+0000","last_became_peered":"2024-09-13T16:52:03.078804+0000","last_unstale":"2024-09-13T16:52:03.081093+0000","last_undegraded":"2024-09-13T16:52:03.081093+0000","last_fullsized":"2024-09-13T16:52:03.081093+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:46.496049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.084516+0000","last_change":"2024-09-13T16:52:02.591479+0000","last_active":"2024-09-13T16:52:03.084516+0000","last_peered":"2024-09-13T16:52:03.084516+0000","last_clean":"2024-09-13T16:52:03.084516+0000","last_became_active":"2024-09-13T16:52:02.591244+0000","last_became_peered":"2024-09-13T16:52:02.591244+0000","last_unstale":"2024-09-13T16:52:03.084516+0000","last_undegraded":"2024-09-13T16:52:03.084516+0000","last_fullsized":"2024-09-13T16:52:03.084516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:30.321115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.083858+0000","last_change":"2024-09-13T16:52:03.080594+0000","last_active":"2024-09-13T16:52:03.083858+0000","last_peered":"2024-09-13T16:52:03.083858+0000","last_clean":"2024-09-13T16:52:03.083858+0000","last_became_active":"2024-09-13T16:52:03.080337+0000","last_became_peered":"2024-09-13T16:52:03.080337+0000","last_unstale":"2024-09-13T16:52:03.083858+0000","last_undegraded":"2024-09-13T16:52:03.083858+0000","last_fullsized":"2024-09-13T16:52:03.083858+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:14.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.068976+0000","last_change":"2024-09-13T16:52:02.586883+0000","last_active":"2024-09-13T16:52:03.068976+0000","last_peered":"2024-09-13T16:52:03.068976+0000","last_clean":"2024-09-13T16:52:03.068976+0000","last_became_active":"2024-09-13T16:52:02.586370+0000","last_became_peered":"2024-09-13T16:52:02.586370+0000","last_unstale":"2024-09-13T16:52:03.068976+0000","last_undegraded":"2024-09-13T16:52:03.068976+0000","last_fullsized":"2024-09-13T16:52:03.068976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:10.154122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068281+0000","last_change":"2024-09-13T16:52:03.067442+0000","last_active":"2024-09-13T16:52:04.068281+0000","last_peered":"2024-09-13T16:52:04.068281+0000","last_clean":"2024-09-13T16:52:04.068281+0000","last_became_active":"2024-09-13T16:52:03.066943+0000","last_became_peered":"2024-09-13T16:52:03.066943+0000","last_unstale":"2024-09-13T16:52:04.068281+0000","last_undegraded":"2024-09-13T16:52:04.068281+0000","last_fullsized":"2024-09-13T16:52:04.068281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:11:21.464886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.067292+0000","last_change":"2024-09-13T16:52:02.583895+0000","last_active":"2024-09-13T16:52:03.067292+0000","last_peered":"2024-09-13T16:52:03.067292+0000","last_clean":"2024-09-13T16:52:03.067292+0000","last_became_active":"2024-09-13T16:52:02.583629+0000","last_became_peered":"2024-09-13T16:52:02.583629+0000","last_unstale":"2024-09-13T16:52:03.067292+0000","last_undegraded":"2024-09-13T16:52:03.067292+0000","last_fullsized":"2024-09-13T16:52:03.067292+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:05.265636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067712+0000","last_change":"2024-09-13T16:52:03.093629+0000","last_active":"2024-09-13T16:52:04.067712+0000","last_peered":"2024-09-13T16:52:04.067712+0000","last_clean":"2024-09-13T16:52:04.067712+0000","last_became_active":"2024-09-13T16:52:03.081106+0000","last_became_peered":"2024-09-13T16:52:03.081106+0000","last_unstale":"2024-09-13T16:52:04.067712+0000","last_undegraded":"2024-09-13T16:52:04.067712+0000","last_fullsized":"2024-09-13T16:52:04.067712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:52.450693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.068821+0000","last_change":"2024-09-13T16:52:02.584869+0000","last_active":"2024-09-13T16:52:03.068821+0000","last_peered":"2024-09-13T16:52:03.068821+0000","last_clean":"2024-09-13T16:52:03.068821+0000","last_became_active":"2024-09-13T16:52:02.584582+0000","last_became_peered":"2024-09-13T16:52:02.584582+0000","last_unstale":"2024-09-13T16:52:03.068821+0000","last_undegraded":"2024-09-13T16:52:03.068821+0000","last_fullsized":"2024-09-13T16:52:03.068821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:53.118849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.091090+0000","last_change":"2024-09-13T16:52:03.082736+0000","last_active":"2024-09-13T16:52:03.091090+0000","last_peered":"2024-09-13T16:52:03.091090+0000","last_clean":"2024-09-13T16:52:03.091090+0000","last_became_active":"2024-09-13T16:52:03.082456+0000","last_became_peered":"2024-09-13T16:52:03.082456+0000","last_unstale":"2024-09-13T16:52:03.091090+0000","last_undegraded":"2024-09-13T16:52:03.091090+0000","last_fullsized":"2024-09-13T16:52:03.091090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:06.994719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069132+0000","last_change":"2024-09-13T16:52:02.593461+0000","last_active":"2024-09-13T16:52:04.069132+0000","last_peered":"2024-09-13T16:52:04.069132+0000","last_clean":"2024-09-13T16:52:04.069132+0000","last_became_active":"2024-09-13T16:52:02.593156+0000","last_became_peered":"2024-09-13T16:52:02.593156+0000","last_unstale":"2024-09-13T16:52:04.069132+0000","last_undegraded":"2024-09-13T16:52:04.069132+0000","last_fullsized":"2024-09-13T16:52:04.069132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:06.193182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.095332+0000","last_change":"2024-09-13T16:52:03.089031+0000","last_active":"2024-09-13T16:52:03.095332+0000","last_peered":"2024-09-13T16:52:03.095332+0000","last_clean":"2024-09-13T16:52:03.095332+0000","last_became_active":"2024-09-13T16:52:03.088746+0000","last_became_peered":"2024-09-13T16:52:03.088746+0000","last_unstale":"2024-09-13T16:52:03.095332+0000","last_undegraded":"2024-09-13T16:52:03.095332+0000","last_fullsized":"2024-09-13T16:52:03.095332+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:32.675438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.077561+0000","last_change":"2024-09-13T16:52:02.588087+0000","last_active":"2024-09-13T16:52:03.077561+0000","last_peered":"2024-09-13T16:52:03.077561+0000","last_clean":"2024-09-13T16:52:03.077561+0000","last_became_active":"2024-09-13T16:52:02.587804+0000","last_became_peered":"2024-09-13T16:52:02.587804+0000","last_unstale":"2024-09-13T16:52:03.077561+0000","last_undegraded":"2024-09-13T16:52:03.077561+0000","last_fullsized":"2024-09-13T16:52:03.077561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:51.707645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068139+0000","last_change":"2024-09-13T16:52:03.100910+0000","last_active":"2024-09-13T16:52:04.068139+0000","last_peered":"2024-09-13T16:52:04.068139+0000","last_clean":"2024-09-13T16:52:04.068139+0000","last_became_active":"2024-09-13T16:52:03.098876+0000","last_became_peered":"2024-09-13T16:52:03.098876+0000","last_unstale":"2024-09-13T16:52:04.068139+0000","last_undegraded":"2024-09-13T16:52:04.068139+0000","last_fullsized":"2024-09-13T16:52:04.068139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:06.544382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.073901+0000","last_change":"2024-09-13T16:52:02.590912+0000","last_active":"2024-09-13T16:52:03.073901+0000","last_peered":"2024-09-13T16:52:03.073901+0000","last_clean":"2024-09-13T16:52:03.073901+0000","last_became_active":"2024-09-13T16:52:02.590498+0000","last_became_peered":"2024-09-13T16:52:02.590498+0000","last_unstale":"2024-09-13T16:52:03.073901+0000","last_undegraded":"2024-09-13T16:52:03.073901+0000","last_fullsized":"2024-09-13T16:52:03.073901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:18.283316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.101443+0000","last_change":"2024-09-13T16:52:03.098763+0000","last_active":"2024-09-13T16:52:03.101443+0000","last_peered":"2024-09-13T16:52:03.101443+0000","last_clean":"2024-09-13T16:52:03.101443+0000","last_became_active":"2024-09-13T16:52:03.098460+0000","last_became_peered":"2024-09-13T16:52:03.098460+0000","last_unstale":"2024-09-13T16:52:03.101443+0000","last_undegraded":"2024-09-13T16:52:03.101443+0000","last_fullsized":"2024-09-13T16:52:03.101443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:27:40.756415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.077710+0000","last_change":"2024-09-13T16:52:02.590814+0000","last_active":"2024-09-13T16:52:03.077710+0000","last_peered":"2024-09-13T16:52:03.077710+0000","last_clean":"2024-09-13T16:52:03.077710+0000","last_became_active":"2024-09-13T16:52:02.590540+0000","last_became_peered":"2024-09-13T16:52:02.590540+0000","last_unstale":"2024-09-13T16:52:03.077710+0000","last_undegraded":"2024-09-13T16:52:03.077710+0000","last_fullsized":"2024-09-13T16:52:03.077710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:45.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.086177+0000","last_change":"2024-09-13T16:52:03.074662+0000","last_active":"2024-09-13T16:52:03.086177+0000","last_peered":"2024-09-13T16:52:03.086177+0000","last_clean":"2024-09-13T16:52:03.086177+0000","last_became_active":"2024-09-13T16:52:03.074189+0000","last_became_peered":"2024-09-13T16:52:03.074189+0000","last_unstale":"2024-09-13T16:52:03.086177+0000","last_undegraded":"2024-09-13T16:52:03.086177+0000","last_fullsized":"2024-09-13T16:52:03.086177+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:40.831264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.064223+0000","last_change":"2024-09-13T16:52:02.585055+0000","last_active":"2024-09-13T16:52:03.064223+0000","last_peered":"2024-09-13T16:52:03.064223+0000","last_clean":"2024-09-13T16:52:03.064223+0000","last_became_active":"2024-09-13T16:52:02.583595+0000","last_became_peered":"2024-09-13T16:52:02.583595+0000","last_unstale":"2024-09-13T16:52:03.064223+0000","last_undegraded":"2024-09-13T16:52:03.064223+0000","last_fullsized":"2024-09-13T16:52:03.064223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:45.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069567+0000","last_change":"2024-09-13T16:52:03.098609+0000","last_active":"2024-09-13T16:52:04.069567+0000","last_peered":"2024-09-13T16:52:04.069567+0000","last_clean":"2024-09-13T16:52:04.069567+0000","last_became_active":"2024-09-13T16:52:03.098374+0000","last_became_peered":"2024-09-13T16:52:03.098374+0000","last_unstale":"2024-09-13T16:52:04.069567+0000","last_undegraded":"2024-09-13T16:52:04.069567+0000","last_fullsized":"2024-09-13T16:52:04.069567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:06.994719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069738+0000","last_change":"2024-09-13T16:52:02.590767+0000","last_active":"2024-09-13T16:52:04.069738+0000","last_peered":"2024-09-13T16:52:04.069738+0000","last_clean":"2024-09-13T16:52:04.069738+0000","last_became_active":"2024-09-13T16:52:02.590437+0000","last_became_peered":"2024-09-13T16:52:02.590437+0000","last_unstale":"2024-09-13T16:52:04.069738+0000","last_undegraded":"2024-09-13T16:52:04.069738+0000","last_fullsized":"2024-09-13T16:52:04.069738+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:06.193182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.092113+0000","last_change":"2024-09-13T16:52:03.084537+0000","last_active":"2024-09-13T16:52:03.092113+0000","last_peered":"2024-09-13T16:52:03.092113+0000","last_clean":"2024-09-13T16:52:03.092113+0000","last_became_active":"2024-09-13T16:52:03.084187+0000","last_became_peered":"2024-09-13T16:52:03.084187+0000","last_unstale":"2024-09-13T16:52:03.092113+0000","last_undegraded":"2024-09-13T16:52:03.092113+0000","last_fullsized":"2024-09-13T16:52:03.092113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:41.160361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067596+0000","last_change":"2024-09-13T16:52:02.596350+0000","last_active":"2024-09-13T16:52:04.067596+0000","last_peered":"2024-09-13T16:52:04.067596+0000","last_clean":"2024-09-13T16:52:04.067596+0000","last_became_active":"2024-09-13T16:52:02.596087+0000","last_became_peered":"2024-09-13T16:52:02.596087+0000","last_unstale":"2024-09-13T16:52:04.067596+0000","last_undegraded":"2024-09-13T16:52:04.067596+0000","last_fullsized":"2024-09-13T16:52:04.067596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:08.524061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069001+0000","last_change":"2024-09-13T16:52:03.083118+0000","last_active":"2024-09-13T16:52:04.069001+0000","last_peered":"2024-09-13T16:52:04.069001+0000","last_clean":"2024-09-13T16:52:04.069001+0000","last_became_active":"2024-09-13T16:52:03.082858+0000","last_became_peered":"2024-09-13T16:52:03.082858+0000","last_unstale":"2024-09-13T16:52:04.069001+0000","last_undegraded":"2024-09-13T16:52:04.069001+0000","last_fullsized":"2024-09-13T16:52:04.069001+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:19.241815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.068848+0000","last_change":"2024-09-13T16:52:02.586740+0000","last_active":"2024-09-13T16:52:03.068848+0000","last_peered":"2024-09-13T16:52:03.068848+0000","last_clean":"2024-09-13T16:52:03.068848+0000","last_became_active":"2024-09-13T16:52:02.586493+0000","last_became_peered":"2024-09-13T16:52:02.586493+0000","last_unstale":"2024-09-13T16:52:03.068848+0000","last_undegraded":"2024-09-13T16:52:03.068848+0000","last_fullsized":"2024-09-13T16:52:03.068848+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:10.154122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T16:52:02.571731+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:52:02.571731+0000","last_peered":"2024-09-13T16:52:02.571731+0000","last_clean":"2024-09-13T16:52:02.571731+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:52:02.571731+0000","last_undegraded":"2024-09-13T16:52:02.571731+0000","last_fullsized":"2024-09-13T16:52:02.571731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1060864,"data_stored":786562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.466}]},{"osd":1,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.083},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.5920000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":5.6959999999999997,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":5.6260000000000003,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.544}]},{"osd":3,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.663}]},{"osd":4,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.581},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.6140000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":3.0139999999999998},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":2.7869999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.5660000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":454656,"data_stored":195142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":3.1739999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.999}]},{"osd":1,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.3320000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.139}]},{"osd":3,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":1.137,"5min":1.137,"15min":1.137},"max":{"1min":3.3769999999999998,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.4950000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.272},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.377}]},{"osd":5,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.6789999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":1.232},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.1779999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":3.2250000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.417}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":458752,"data_stored":194672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.175},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.2}]},{"osd":1,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":1.026,"5min":1.026,"15min":1.026},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.23}]},{"osd":2,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.405}]},{"osd":3,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.8300000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.0780000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.304},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.534}]},{"osd":6,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.4990000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":3.5630000000000002,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.2809999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":471040,"data_stored":201784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.3280000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.411},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.6080000000000001,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.5169999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":4.8099999999999996,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.125},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.204},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.268}]},{"osd":7,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.579,"15min":1.579},"min":{"1min":1.107,"5min":1.107,"15min":1.107},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":2.6619999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.468}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":196158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.175}]},{"osd":1,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.129}]},{"osd":4,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.339},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.23}]},{"osd":5,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.397}]},{"osd":6,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.466}]},{"osd":7,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.3600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.331}]},{"osd":1,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.012}]},{"osd":3,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.137}]},{"osd":4,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.1930000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.5469999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"min":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":1.073,"5min":1.073,"15min":1.073},"max":{"1min":3.3980000000000001,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.637},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"max":{"1min":3.4060000000000001,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.2589999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0580000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.7610000000000001,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.345},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.9630000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.0189999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":2.7440000000000002,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.2629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":784962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.0149999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.5430000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.147}]},{"osd":4,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.1879999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":3.3999999999999999,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":1.383},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.9860000000000002,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.696}]},{"osd":6,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"max":{"1min":3.3039999999999998,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":1.466}]},{"osd":7,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.7789999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:52:07.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:52:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.648+0000 7f4efd000700 1 Processor -- start 2024-09-13T16:52:07.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.652+0000 7f4efd000700 1 -- start start 2024-09-13T16:52:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.652+0000 7f4efd000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ef815b660 0x7f4ef80b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.652+0000 7f4efd000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef80b2f40 0x7f4ef8070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.652+0000 7f4efd000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ef80712f0 0x7f4ef8164d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.652+0000 7f4efd000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef80b40d0 con 0x7f4ef80712f0 2024-09-13T16:52:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.652+0000 7f4efd000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef8071780 con 0x7f4ef815b660 2024-09-13T16:52:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.652+0000 7f4efd000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef80b3450 con 0x7f4ef80b2f40 2024-09-13T16:52:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef80b2f40 0x7f4ef8070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4ef6800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ef80712f0 0x7f4ef8164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4ef5e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ef815b660 0x7f4ef80b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef80b2f40 0x7f4ef8070db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:47674/0 (socket says 172.21.15.3:47674) 2024-09-13T16:52:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 -- 172.21.15.3:0/1075780512 learned_addr learned my addr 172.21.15.3:0/1075780512 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 -- 172.21.15.3:0/1075780512 >> v2:172.21.15.157:3300/0 conn(0x7f4ef815b660 msgr2=0x7f4ef80b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 --2- 172.21.15.3:0/1075780512 >> v2:172.21.15.157:3300/0 conn(0x7f4ef815b660 0x7f4ef80b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 -- 172.21.15.3:0/1075780512 >> v2:172.21.15.3:3300/0 conn(0x7f4ef80712f0 msgr2=0x7f4ef8164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 --2- 172.21.15.3:0/1075780512 >> v2:172.21.15.3:3300/0 conn(0x7f4ef80712f0 0x7f4ef8164d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 -- 172.21.15.3:0/1075780512 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef80b3730 con 0x7f4ef80b2f40 2024-09-13T16:52:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4ef6800700 1 --2- 172.21.15.3:0/1075780512 >> v2:172.21.15.3:3300/0 conn(0x7f4ef80712f0 0x7f4ef8164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:07.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4eed400700 1 --2- 172.21.15.3:0/1075780512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef80b2f40 0x7f4ef8070db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ef806e080 tx=0x7f4ee0009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b81fe4078afe49b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4ef5400700 1 -- 172.21.15.3:0/1075780512 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0011830 con 0x7f4ef80b2f40 2024-09-13T16:52:07.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4ef5400700 1 -- 172.21.15.3:0/1075780512 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee0011990 con 0x7f4ef80b2f40 2024-09-13T16:52:07.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4ef5400700 1 -- 172.21.15.3:0/1075780512 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0011d20 con 0x7f4ef80b2f40 2024-09-13T16:52:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4efd000700 1 -- 172.21.15.3:0/1075780512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef80b2f40 msgr2=0x7f4ef8070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4efd000700 1 --2- 172.21.15.3:0/1075780512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef80b2f40 0x7f4ef8070db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ef806e080 tx=0x7f4ee0009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4efd000700 1 -- 172.21.15.3:0/1075780512 shutdown_connections 2024-09-13T16:52:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4efd000700 1 --2- 172.21.15.3:0/1075780512 >> v2:172.21.15.3:3300/0 conn(0x7f4ef80712f0 0x7f4ef8164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4efd000700 1 --2- 172.21.15.3:0/1075780512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef80b2f40 0x7f4ef8070db0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4efd000700 1 --2- 172.21.15.3:0/1075780512 >> v2:172.21.15.157:3300/0 conn(0x7f4ef815b660 0x7f4ef80b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4efd000700 1 -- 172.21.15.3:0/1075780512 >> 172.21.15.3:0/1075780512 conn(0x7f4ef806a390 msgr2=0x7f4ef806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.656+0000 7f4efd000700 1 -- 172.21.15.3:0/1075780512 shutdown_connections 2024-09-13T16:52:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.660+0000 7f4efd000700 1 -- 172.21.15.3:0/1075780512 wait complete. 2024-09-13T16:52:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.660+0000 7f4efd000700 1 Processor -- start 2024-09-13T16:52:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.660+0000 7f4efd000700 1 -- start start 2024-09-13T16:52:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4efd000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ef80712f0 0x7f4ef8142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4efd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef80b2f40 0x7f4ef8159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4efd000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef815b660 0x7f4ef81441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4efd000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8143dc0 con 0x7f4ef80712f0 2024-09-13T16:52:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4efd000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef81428d0 con 0x7f4ef80b2f40 2024-09-13T16:52:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4efd000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef815a500 con 0x7f4ef815b660 2024-09-13T16:52:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ef80712f0 0x7f4ef8142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ef80712f0 0x7f4ef8142b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44944/0 (socket says 172.21.15.3:44944) 2024-09-13T16:52:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 -- 172.21.15.3:0/3966973271 learned_addr learned my addr 172.21.15.3:0/3966973271 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef6800700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef815b660 0x7f4ef81441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4eed400700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef80b2f40 0x7f4ef8159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 -- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef815b660 msgr2=0x7f4ef81441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef815b660 0x7f4ef81441e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 -- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef80b2f40 msgr2=0x7f4ef8159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef80b2f40 0x7f4ef8159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 -- 172.21.15.3:0/3966973271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef00094b0 con 0x7f4ef80712f0 2024-09-13T16:52:07.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4ef5e00700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ef80712f0 0x7f4ef8142b60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8007e70 tx=0x7f4ee800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4eeea00700 1 -- 172.21.15.3:0/3966973271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800ed30 con 0x7f4ef80712f0 2024-09-13T16:52:07.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4eeea00700 1 -- 172.21.15.3:0/3966973271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee800ee90 con 0x7f4ef80712f0 2024-09-13T16:52:07.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee0009570 con 0x7f4ef80712f0 2024-09-13T16:52:07.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4eeea00700 1 -- 172.21.15.3:0/3966973271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8014660 con 0x7f4ef80712f0 2024-09-13T16:52:07.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.664+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef8143140 con 0x7f4ef80712f0 2024-09-13T16:52:07.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.668+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef8144030 con 0x7f4ef80712f0 2024-09-13T16:52:07.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.668+0000 7f4eeea00700 1 -- 172.21.15.3:0/3966973271 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ee80147c0 con 0x7f4ef80712f0 2024-09-13T16:52:07.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.668+0000 7f4eeea00700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4ed4066730 0x7f4ed4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.668+0000 7f4eeea00700 1 -- 172.21.15.3:0/3966973271 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4ee8084970 con 0x7f4ef80712f0 2024-09-13T16:52:07.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.672+0000 7f4eed400700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4ed4066730 0x7f4ed4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.672+0000 7f4eed400700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4ed4066730 0x7f4ed4068bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80b3380 tx=0x7f4ee001d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.676+0000 7f4eeea00700 1 -- 172.21.15.3:0/3966973271 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ee8051b00 con 0x7f4ef80712f0 2024-09-13T16:52:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.860+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ef80b3cf0 con 0x7f4ed4066730 2024-09-13T16:52:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.880+0000 7f4eeea00700 1 -- 172.21.15.3:0/3966973271 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287057 (secure 0 0 0) 0x7f4ee0011630 con 0x7f4ed4066730 2024-09-13T16:52:07.890 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4ed4066730 msgr2=0x7f4ed4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4ed4066730 0x7f4ed4068bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80b3380 tx=0x7f4ee001d040 comp rx=0 tx=0).stop 2024-09-13T16:52:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ef80712f0 msgr2=0x7f4ef8142b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ef80712f0 0x7f4ef8142b60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8007e70 tx=0x7f4ee800e3c0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 shutdown_connections 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4ed4066730 0x7f4ed4068bf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ef815b660 0x7f4ef81441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef80b2f40 0x7f4ef8159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 --2- 172.21.15.3:0/3966973271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ef80712f0 0x7f4ef8142b60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 >> 172.21.15.3:0/3966973271 conn(0x7f4ef806a390 msgr2=0x7f4ef80cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 shutdown_connections 2024-09-13T16:52:07.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:07.888+0000 7f4efd000700 1 -- 172.21.15.3:0/3966973271 wait complete. 2024-09-13T16:52:07.895 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:52:07.916 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-13T16:52:07.229588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237328,"kb_used_data":4756,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737392,"statfs":{"total":773094113280,"available":772851089408,"internally_reserved":0,"allocated":4870144,"data_stored":2748560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.674890"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068248+0000","last_change":"2024-09-13T16:52:03.080863+0000","last_active":"2024-09-13T16:52:04.068248+0000","last_peered":"2024-09-13T16:52:04.068248+0000","last_clean":"2024-09-13T16:52:04.068248+0000","last_became_active":"2024-09-13T16:52:03.080642+0000","last_became_peered":"2024-09-13T16:52:03.080642+0000","last_unstale":"2024-09-13T16:52:04.068248+0000","last_undegraded":"2024-09-13T16:52:04.068248+0000","last_fullsized":"2024-09-13T16:52:04.068248+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:36.778123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067632+0000","last_change":"2024-09-13T16:52:03.092688+0000","last_active":"2024-09-13T16:52:04.067632+0000","last_peered":"2024-09-13T16:52:04.067632+0000","last_clean":"2024-09-13T16:52:04.067632+0000","last_became_active":"2024-09-13T16:52:03.092437+0000","last_became_peered":"2024-09-13T16:52:03.092437+0000","last_unstale":"2024-09-13T16:52:04.067632+0000","last_undegraded":"2024-09-13T16:52:04.067632+0000","last_fullsized":"2024-09-13T16:52:04.067632+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:56.542780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.085110+0000","last_change":"2024-09-13T16:52:03.082017+0000","last_active":"2024-09-13T16:52:03.085110+0000","last_peered":"2024-09-13T16:52:03.085110+0000","last_clean":"2024-09-13T16:52:03.085110+0000","last_became_active":"2024-09-13T16:52:03.081747+0000","last_became_peered":"2024-09-13T16:52:03.081747+0000","last_unstale":"2024-09-13T16:52:03.085110+0000","last_undegraded":"2024-09-13T16:52:03.085110+0000","last_fullsized":"2024-09-13T16:52:03.085110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:11.241424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.089846+0000","last_change":"2024-09-13T16:52:03.086629+0000","last_active":"2024-09-13T16:52:03.089846+0000","last_peered":"2024-09-13T16:52:03.089846+0000","last_clean":"2024-09-13T16:52:03.089846+0000","last_became_active":"2024-09-13T16:52:03.086364+0000","last_became_peered":"2024-09-13T16:52:03.086364+0000","last_unstale":"2024-09-13T16:52:03.089846+0000","last_undegraded":"2024-09-13T16:52:03.089846+0000","last_fullsized":"2024-09-13T16:52:03.089846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:55.865168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.091626+0000","last_change":"2024-09-13T16:52:03.085436+0000","last_active":"2024-09-13T16:52:03.091626+0000","last_peered":"2024-09-13T16:52:03.091626+0000","last_clean":"2024-09-13T16:52:03.091626+0000","last_became_active":"2024-09-13T16:52:03.085189+0000","last_became_peered":"2024-09-13T16:52:03.085189+0000","last_unstale":"2024-09-13T16:52:03.091626+0000","last_undegraded":"2024-09-13T16:52:03.091626+0000","last_fullsized":"2024-09-13T16:52:03.091626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:19.293991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.100240+0000","last_change":"2024-09-13T16:52:03.098673+0000","last_active":"2024-09-13T16:52:03.100240+0000","last_peered":"2024-09-13T16:52:03.100240+0000","last_clean":"2024-09-13T16:52:03.100240+0000","last_became_active":"2024-09-13T16:52:03.098431+0000","last_became_peered":"2024-09-13T16:52:03.098431+0000","last_unstale":"2024-09-13T16:52:03.100240+0000","last_undegraded":"2024-09-13T16:52:03.100240+0000","last_fullsized":"2024-09-13T16:52:03.100240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:11.241424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068232+0000","last_change":"2024-09-13T16:52:03.081059+0000","last_active":"2024-09-13T16:52:04.068232+0000","last_peered":"2024-09-13T16:52:04.068232+0000","last_clean":"2024-09-13T16:52:04.068232+0000","last_became_active":"2024-09-13T16:52:03.080623+0000","last_became_peered":"2024-09-13T16:52:03.080623+0000","last_unstale":"2024-09-13T16:52:04.068232+0000","last_undegraded":"2024-09-13T16:52:04.068232+0000","last_fullsized":"2024-09-13T16:52:04.068232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:48.651139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067490+0000","last_change":"2024-09-13T16:52:03.092917+0000","last_active":"2024-09-13T16:52:04.067490+0000","last_peered":"2024-09-13T16:52:04.067490+0000","last_clean":"2024-09-13T16:52:04.067490+0000","last_became_active":"2024-09-13T16:52:03.083916+0000","last_became_peered":"2024-09-13T16:52:03.083916+0000","last_unstale":"2024-09-13T16:52:04.067490+0000","last_undegraded":"2024-09-13T16:52:04.067490+0000","last_fullsized":"2024-09-13T16:52:04.067490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:02.499074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.090179+0000","last_change":"2024-09-13T16:52:03.089092+0000","last_active":"2024-09-13T16:52:03.090179+0000","last_peered":"2024-09-13T16:52:03.090179+0000","last_clean":"2024-09-13T16:52:03.090179+0000","last_became_active":"2024-09-13T16:52:03.088825+0000","last_became_peered":"2024-09-13T16:52:03.088825+0000","last_unstale":"2024-09-13T16:52:03.090179+0000","last_undegraded":"2024-09-13T16:52:03.090179+0000","last_fullsized":"2024-09-13T16:52:03.090179+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:06.994719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068775+0000","last_change":"2024-09-13T16:52:03.072425+0000","last_active":"2024-09-13T16:52:04.068775+0000","last_peered":"2024-09-13T16:52:04.068775+0000","last_clean":"2024-09-13T16:52:04.068775+0000","last_became_active":"2024-09-13T16:52:03.072102+0000","last_became_peered":"2024-09-13T16:52:03.072102+0000","last_unstale":"2024-09-13T16:52:04.068775+0000","last_undegraded":"2024-09-13T16:52:04.068775+0000","last_fullsized":"2024-09-13T16:52:04.068775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:06.544382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069267+0000","last_change":"2024-09-13T16:52:03.086328+0000","last_active":"2024-09-13T16:52:04.069267+0000","last_peered":"2024-09-13T16:52:04.069267+0000","last_clean":"2024-09-13T16:52:04.069267+0000","last_became_active":"2024-09-13T16:52:03.086079+0000","last_became_peered":"2024-09-13T16:52:03.086079+0000","last_unstale":"2024-09-13T16:52:04.069267+0000","last_undegraded":"2024-09-13T16:52:04.069267+0000","last_fullsized":"2024-09-13T16:52:04.069267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:46.850446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.083253+0000","last_change":"2024-09-13T16:52:03.078958+0000","last_active":"2024-09-13T16:52:03.083253+0000","last_peered":"2024-09-13T16:52:03.083253+0000","last_clean":"2024-09-13T16:52:03.083253+0000","last_became_active":"2024-09-13T16:52:03.078698+0000","last_became_peered":"2024-09-13T16:52:03.078698+0000","last_unstale":"2024-09-13T16:52:03.083253+0000","last_undegraded":"2024-09-13T16:52:03.083253+0000","last_fullsized":"2024-09-13T16:52:03.083253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:40.831264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068478+0000","last_change":"2024-09-13T16:52:03.078315+0000","last_active":"2024-09-13T16:52:04.068478+0000","last_peered":"2024-09-13T16:52:04.068478+0000","last_clean":"2024-09-13T16:52:04.068478+0000","last_became_active":"2024-09-13T16:52:03.074000+0000","last_became_peered":"2024-09-13T16:52:03.074000+0000","last_unstale":"2024-09-13T16:52:04.068478+0000","last_undegraded":"2024-09-13T16:52:04.068478+0000","last_fullsized":"2024-09-13T16:52:04.068478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:48.611064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069532+0000","last_change":"2024-09-13T16:52:03.090755+0000","last_active":"2024-09-13T16:52:04.069532+0000","last_peered":"2024-09-13T16:52:04.069532+0000","last_clean":"2024-09-13T16:52:04.069532+0000","last_became_active":"2024-09-13T16:52:03.090511+0000","last_became_peered":"2024-09-13T16:52:03.090511+0000","last_unstale":"2024-09-13T16:52:04.069532+0000","last_undegraded":"2024-09-13T16:52:04.069532+0000","last_fullsized":"2024-09-13T16:52:04.069532+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:11.241424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068450+0000","last_change":"2024-09-13T16:52:03.080866+0000","last_active":"2024-09-13T16:52:04.068450+0000","last_peered":"2024-09-13T16:52:04.068450+0000","last_clean":"2024-09-13T16:52:04.068450+0000","last_became_active":"2024-09-13T16:52:03.080264+0000","last_became_peered":"2024-09-13T16:52:03.080264+0000","last_unstale":"2024-09-13T16:52:04.068450+0000","last_undegraded":"2024-09-13T16:52:04.068450+0000","last_fullsized":"2024-09-13T16:52:04.068450+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:09.534736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.094881+0000","last_change":"2024-09-13T16:52:03.088360+0000","last_active":"2024-09-13T16:52:03.094881+0000","last_peered":"2024-09-13T16:52:03.094881+0000","last_clean":"2024-09-13T16:52:03.094881+0000","last_became_active":"2024-09-13T16:52:03.084442+0000","last_became_peered":"2024-09-13T16:52:03.084442+0000","last_unstale":"2024-09-13T16:52:03.094881+0000","last_undegraded":"2024-09-13T16:52:03.094881+0000","last_fullsized":"2024-09-13T16:52:03.094881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:18.226039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.095941+0000","last_change":"2024-09-13T16:52:03.093441+0000","last_active":"2024-09-13T16:52:03.095941+0000","last_peered":"2024-09-13T16:52:03.095941+0000","last_clean":"2024-09-13T16:52:03.095941+0000","last_became_active":"2024-09-13T16:52:03.093113+0000","last_became_peered":"2024-09-13T16:52:03.093113+0000","last_unstale":"2024-09-13T16:52:03.095941+0000","last_undegraded":"2024-09-13T16:52:03.095941+0000","last_fullsized":"2024-09-13T16:52:03.095941+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:14.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.081864+0000","last_change":"2024-09-13T16:52:03.076797+0000","last_active":"2024-09-13T16:52:03.081864+0000","last_peered":"2024-09-13T16:52:03.081864+0000","last_clean":"2024-09-13T16:52:03.081864+0000","last_became_active":"2024-09-13T16:52:03.076495+0000","last_became_peered":"2024-09-13T16:52:03.076495+0000","last_unstale":"2024-09-13T16:52:03.081864+0000","last_undegraded":"2024-09-13T16:52:03.081864+0000","last_fullsized":"2024-09-13T16:52:03.081864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:47.136150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.090490+0000","last_change":"2024-09-13T16:52:03.085759+0000","last_active":"2024-09-13T16:52:03.090490+0000","last_peered":"2024-09-13T16:52:03.090490+0000","last_clean":"2024-09-13T16:52:03.090490+0000","last_became_active":"2024-09-13T16:52:03.085511+0000","last_became_peered":"2024-09-13T16:52:03.085511+0000","last_unstale":"2024-09-13T16:52:03.090490+0000","last_undegraded":"2024-09-13T16:52:03.090490+0000","last_fullsized":"2024-09-13T16:52:03.090490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:41.160361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.096134+0000","last_change":"2024-09-13T16:52:03.091312+0000","last_active":"2024-09-13T16:52:03.096134+0000","last_peered":"2024-09-13T16:52:03.096134+0000","last_clean":"2024-09-13T16:52:03.096134+0000","last_became_active":"2024-09-13T16:52:03.091043+0000","last_became_peered":"2024-09-13T16:52:03.091043+0000","last_unstale":"2024-09-13T16:52:03.096134+0000","last_undegraded":"2024-09-13T16:52:03.096134+0000","last_fullsized":"2024-09-13T16:52:03.096134+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:29.500537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068232+0000","last_change":"2024-09-13T16:52:03.085790+0000","last_active":"2024-09-13T16:52:04.068232+0000","last_peered":"2024-09-13T16:52:04.068232+0000","last_clean":"2024-09-13T16:52:04.068232+0000","last_became_active":"2024-09-13T16:52:03.083654+0000","last_became_peered":"2024-09-13T16:52:03.083654+0000","last_unstale":"2024-09-13T16:52:04.068232+0000","last_undegraded":"2024-09-13T16:52:04.068232+0000","last_fullsized":"2024-09-13T16:52:04.068232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:09.761884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069986+0000","last_change":"2024-09-13T16:52:03.095433+0000","last_active":"2024-09-13T16:52:04.069986+0000","last_peered":"2024-09-13T16:52:04.069986+0000","last_clean":"2024-09-13T16:52:04.069986+0000","last_became_active":"2024-09-13T16:52:03.095122+0000","last_became_peered":"2024-09-13T16:52:03.095122+0000","last_unstale":"2024-09-13T16:52:04.069986+0000","last_undegraded":"2024-09-13T16:52:04.069986+0000","last_fullsized":"2024-09-13T16:52:04.069986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:10.717561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.092998+0000","last_change":"2024-09-13T16:52:03.082641+0000","last_active":"2024-09-13T16:52:03.092998+0000","last_peered":"2024-09-13T16:52:03.092998+0000","last_clean":"2024-09-13T16:52:03.092998+0000","last_became_active":"2024-09-13T16:52:03.082166+0000","last_became_peered":"2024-09-13T16:52:03.082166+0000","last_unstale":"2024-09-13T16:52:03.092998+0000","last_undegraded":"2024-09-13T16:52:03.092998+0000","last_fullsized":"2024-09-13T16:52:03.092998+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:33.230927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.077021+0000","last_change":"2024-09-13T16:52:03.071454+0000","last_active":"2024-09-13T16:52:03.077021+0000","last_peered":"2024-09-13T16:52:03.077021+0000","last_clean":"2024-09-13T16:52:03.077021+0000","last_became_active":"2024-09-13T16:52:03.071161+0000","last_became_peered":"2024-09-13T16:52:03.071161+0000","last_unstale":"2024-09-13T16:52:03.077021+0000","last_undegraded":"2024-09-13T16:52:03.077021+0000","last_fullsized":"2024-09-13T16:52:03.077021+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:29.699627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.093641+0000","last_change":"2024-09-13T16:52:03.090404+0000","last_active":"2024-09-13T16:52:03.093641+0000","last_peered":"2024-09-13T16:52:03.093641+0000","last_clean":"2024-09-13T16:52:03.093641+0000","last_became_active":"2024-09-13T16:52:03.082870+0000","last_became_peered":"2024-09-13T16:52:03.082870+0000","last_unstale":"2024-09-13T16:52:03.093641+0000","last_undegraded":"2024-09-13T16:52:03.093641+0000","last_fullsized":"2024-09-13T16:52:03.093641+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:40.831264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.090580+0000","last_change":"2024-09-13T16:52:03.080280+0000","last_active":"2024-09-13T16:52:03.090580+0000","last_peered":"2024-09-13T16:52:03.090580+0000","last_clean":"2024-09-13T16:52:03.090580+0000","last_became_active":"2024-09-13T16:52:03.080020+0000","last_became_peered":"2024-09-13T16:52:03.080020+0000","last_unstale":"2024-09-13T16:52:03.090580+0000","last_undegraded":"2024-09-13T16:52:03.090580+0000","last_fullsized":"2024-09-13T16:52:03.090580+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:27:40.756415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.082430+0000","last_change":"2024-09-13T16:52:03.077877+0000","last_active":"2024-09-13T16:52:03.082430+0000","last_peered":"2024-09-13T16:52:03.082430+0000","last_clean":"2024-09-13T16:52:03.082430+0000","last_became_active":"2024-09-13T16:52:03.077637+0000","last_became_peered":"2024-09-13T16:52:03.077637+0000","last_unstale":"2024-09-13T16:52:03.082430+0000","last_undegraded":"2024-09-13T16:52:03.082430+0000","last_fullsized":"2024-09-13T16:52:03.082430+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:18.226039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068773+0000","last_change":"2024-09-13T16:52:03.073636+0000","last_active":"2024-09-13T16:52:04.068773+0000","last_peered":"2024-09-13T16:52:04.068773+0000","last_clean":"2024-09-13T16:52:04.068773+0000","last_became_active":"2024-09-13T16:52:03.073012+0000","last_became_peered":"2024-09-13T16:52:03.073012+0000","last_unstale":"2024-09-13T16:52:04.068773+0000","last_undegraded":"2024-09-13T16:52:04.068773+0000","last_fullsized":"2024-09-13T16:52:04.068773+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:41.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.081718+0000","last_change":"2024-09-13T16:52:03.075069+0000","last_active":"2024-09-13T16:52:03.081718+0000","last_peered":"2024-09-13T16:52:03.081718+0000","last_clean":"2024-09-13T16:52:03.081718+0000","last_became_active":"2024-09-13T16:52:03.074351+0000","last_became_peered":"2024-09-13T16:52:03.074351+0000","last_unstale":"2024-09-13T16:52:03.081718+0000","last_undegraded":"2024-09-13T16:52:03.081718+0000","last_fullsized":"2024-09-13T16:52:03.081718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:14.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.082038+0000","last_change":"2024-09-13T16:52:03.077430+0000","last_active":"2024-09-13T16:52:03.082038+0000","last_peered":"2024-09-13T16:52:03.082038+0000","last_clean":"2024-09-13T16:52:03.082038+0000","last_became_active":"2024-09-13T16:52:03.077177+0000","last_became_peered":"2024-09-13T16:52:03.077177+0000","last_unstale":"2024-09-13T16:52:03.082038+0000","last_undegraded":"2024-09-13T16:52:03.082038+0000","last_fullsized":"2024-09-13T16:52:03.082038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:05.911241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.093017+0000","last_change":"2024-09-13T16:52:03.083046+0000","last_active":"2024-09-13T16:52:03.093017+0000","last_peered":"2024-09-13T16:52:03.093017+0000","last_clean":"2024-09-13T16:52:03.093017+0000","last_became_active":"2024-09-13T16:52:03.082804+0000","last_became_peered":"2024-09-13T16:52:03.082804+0000","last_unstale":"2024-09-13T16:52:03.093017+0000","last_undegraded":"2024-09-13T16:52:03.093017+0000","last_fullsized":"2024-09-13T16:52:03.093017+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:53.962488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.086787+0000","last_change":"2024-09-13T16:52:03.077907+0000","last_active":"2024-09-13T16:52:03.086787+0000","last_peered":"2024-09-13T16:52:03.086787+0000","last_clean":"2024-09-13T16:52:03.086787+0000","last_became_active":"2024-09-13T16:52:03.077631+0000","last_became_peered":"2024-09-13T16:52:03.077631+0000","last_unstale":"2024-09-13T16:52:03.086787+0000","last_undegraded":"2024-09-13T16:52:03.086787+0000","last_fullsized":"2024-09-13T16:52:03.086787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:58.383411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.094495+0000","last_change":"2024-09-13T16:52:03.087742+0000","last_active":"2024-09-13T16:52:03.094495+0000","last_peered":"2024-09-13T16:52:03.094495+0000","last_clean":"2024-09-13T16:52:03.094495+0000","last_became_active":"2024-09-13T16:52:03.087514+0000","last_became_peered":"2024-09-13T16:52:03.087514+0000","last_unstale":"2024-09-13T16:52:03.094495+0000","last_undegraded":"2024-09-13T16:52:03.094495+0000","last_fullsized":"2024-09-13T16:52:03.094495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:05.911241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.097597+0000","last_change":"2024-09-13T16:52:02.585632+0000","last_active":"2024-09-13T16:52:03.097597+0000","last_peered":"2024-09-13T16:52:03.097597+0000","last_clean":"2024-09-13T16:52:03.097597+0000","last_became_active":"2024-09-13T16:52:02.585369+0000","last_became_peered":"2024-09-13T16:52:02.585369+0000","last_unstale":"2024-09-13T16:52:03.097597+0000","last_undegraded":"2024-09-13T16:52:03.097597+0000","last_fullsized":"2024-09-13T16:52:03.097597+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:14.240207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.090881+0000","last_change":"2024-09-13T16:52:03.084507+0000","last_active":"2024-09-13T16:52:03.090881+0000","last_peered":"2024-09-13T16:52:03.090881+0000","last_clean":"2024-09-13T16:52:03.090881+0000","last_became_active":"2024-09-13T16:52:03.084213+0000","last_became_peered":"2024-09-13T16:52:03.084213+0000","last_unstale":"2024-09-13T16:52:03.090881+0000","last_undegraded":"2024-09-13T16:52:03.090881+0000","last_fullsized":"2024-09-13T16:52:03.090881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:19.665369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.095015+0000","last_change":"2024-09-13T16:52:02.583744+0000","last_active":"2024-09-13T16:52:03.095015+0000","last_peered":"2024-09-13T16:52:03.095015+0000","last_clean":"2024-09-13T16:52:03.095015+0000","last_became_active":"2024-09-13T16:52:02.583472+0000","last_became_peered":"2024-09-13T16:52:02.583472+0000","last_unstale":"2024-09-13T16:52:03.095015+0000","last_undegraded":"2024-09-13T16:52:03.095015+0000","last_fullsized":"2024-09-13T16:52:03.095015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:40.149686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069994+0000","last_change":"2024-09-13T16:52:03.088688+0000","last_active":"2024-09-13T16:52:04.069994+0000","last_peered":"2024-09-13T16:52:04.069994+0000","last_clean":"2024-09-13T16:52:04.069994+0000","last_became_active":"2024-09-13T16:52:03.088276+0000","last_became_peered":"2024-09-13T16:52:03.088276+0000","last_unstale":"2024-09-13T16:52:04.069994+0000","last_undegraded":"2024-09-13T16:52:04.069994+0000","last_fullsized":"2024-09-13T16:52:04.069994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:27:40.756415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.099194+0000","last_change":"2024-09-13T16:52:02.587230+0000","last_active":"2024-09-13T16:52:03.099194+0000","last_peered":"2024-09-13T16:52:03.099194+0000","last_clean":"2024-09-13T16:52:03.099194+0000","last_became_active":"2024-09-13T16:52:02.586966+0000","last_became_peered":"2024-09-13T16:52:02.586966+0000","last_unstale":"2024-09-13T16:52:03.099194+0000","last_undegraded":"2024-09-13T16:52:03.099194+0000","last_fullsized":"2024-09-13T16:52:03.099194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:40.149686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068196+0000","last_change":"2024-09-13T16:52:03.100886+0000","last_active":"2024-09-13T16:52:04.068196+0000","last_peered":"2024-09-13T16:52:04.068196+0000","last_clean":"2024-09-13T16:52:04.068196+0000","last_became_active":"2024-09-13T16:52:03.097629+0000","last_became_peered":"2024-09-13T16:52:03.097629+0000","last_unstale":"2024-09-13T16:52:04.068196+0000","last_undegraded":"2024-09-13T16:52:04.068196+0000","last_fullsized":"2024-09-13T16:52:04.068196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:14.150372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069819+0000","last_change":"2024-09-13T16:52:02.591784+0000","last_active":"2024-09-13T16:52:04.069819+0000","last_peered":"2024-09-13T16:52:04.069819+0000","last_clean":"2024-09-13T16:52:04.069819+0000","last_became_active":"2024-09-13T16:52:02.591544+0000","last_became_peered":"2024-09-13T16:52:02.591544+0000","last_unstale":"2024-09-13T16:52:04.069819+0000","last_undegraded":"2024-09-13T16:52:04.069819+0000","last_fullsized":"2024-09-13T16:52:04.069819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:45.332315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.089747+0000","last_change":"2024-09-13T16:52:03.087733+0000","last_active":"2024-09-13T16:52:03.089747+0000","last_peered":"2024-09-13T16:52:03.089747+0000","last_clean":"2024-09-13T16:52:03.089747+0000","last_became_active":"2024-09-13T16:52:03.087269+0000","last_became_peered":"2024-09-13T16:52:03.087269+0000","last_unstale":"2024-09-13T16:52:03.089747+0000","last_undegraded":"2024-09-13T16:52:03.089747+0000","last_fullsized":"2024-09-13T16:52:03.089747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:19.665369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068350+0000","last_change":"2024-09-13T16:52:02.596848+0000","last_active":"2024-09-13T16:52:04.068350+0000","last_peered":"2024-09-13T16:52:04.068350+0000","last_clean":"2024-09-13T16:52:04.068350+0000","last_became_active":"2024-09-13T16:52:02.596572+0000","last_became_peered":"2024-09-13T16:52:02.596572+0000","last_unstale":"2024-09-13T16:52:04.068350+0000","last_undegraded":"2024-09-13T16:52:04.068350+0000","last_fullsized":"2024-09-13T16:52:04.068350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:31.664763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068479+0000","last_change":"2024-09-13T16:52:03.074248+0000","last_active":"2024-09-13T16:52:04.068479+0000","last_peered":"2024-09-13T16:52:04.068479+0000","last_clean":"2024-09-13T16:52:04.068479+0000","last_became_active":"2024-09-13T16:52:03.073922+0000","last_became_peered":"2024-09-13T16:52:03.073922+0000","last_unstale":"2024-09-13T16:52:04.068479+0000","last_undegraded":"2024-09-13T16:52:04.068479+0000","last_fullsized":"2024-09-13T16:52:04.068479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:06.994719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.095741+0000","last_change":"2024-09-13T16:52:02.584720+0000","last_active":"2024-09-13T16:52:03.095741+0000","last_peered":"2024-09-13T16:52:03.095741+0000","last_clean":"2024-09-13T16:52:03.095741+0000","last_became_active":"2024-09-13T16:52:02.584483+0000","last_became_peered":"2024-09-13T16:52:02.584483+0000","last_unstale":"2024-09-13T16:52:03.095741+0000","last_undegraded":"2024-09-13T16:52:03.095741+0000","last_fullsized":"2024-09-13T16:52:03.095741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:53.118849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069790+0000","last_change":"2024-09-13T16:52:03.084520+0000","last_active":"2024-09-13T16:52:04.069790+0000","last_peered":"2024-09-13T16:52:04.069790+0000","last_clean":"2024-09-13T16:52:04.069790+0000","last_became_active":"2024-09-13T16:52:03.084128+0000","last_became_peered":"2024-09-13T16:52:03.084128+0000","last_unstale":"2024-09-13T16:52:04.069790+0000","last_undegraded":"2024-09-13T16:52:04.069790+0000","last_fullsized":"2024-09-13T16:52:04.069790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:48.651139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.074879+0000","last_change":"2024-09-13T16:52:02.585288+0000","last_active":"2024-09-13T16:52:03.074879+0000","last_peered":"2024-09-13T16:52:03.074879+0000","last_clean":"2024-09-13T16:52:03.074879+0000","last_became_active":"2024-09-13T16:52:02.583943+0000","last_became_peered":"2024-09-13T16:52:02.583943+0000","last_unstale":"2024-09-13T16:52:03.074879+0000","last_undegraded":"2024-09-13T16:52:03.074879+0000","last_fullsized":"2024-09-13T16:52:03.074879+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:45.332315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067662+0000","last_change":"2024-09-13T16:52:03.082320+0000","last_active":"2024-09-13T16:52:04.067662+0000","last_peered":"2024-09-13T16:52:04.067662+0000","last_clean":"2024-09-13T16:52:04.067662+0000","last_became_active":"2024-09-13T16:52:03.080798+0000","last_became_peered":"2024-09-13T16:52:03.080798+0000","last_unstale":"2024-09-13T16:52:04.067662+0000","last_undegraded":"2024-09-13T16:52:04.067662+0000","last_fullsized":"2024-09-13T16:52:04.067662+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:30.022521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069333+0000","last_change":"2024-09-13T16:52:02.586484+0000","last_active":"2024-09-13T16:52:04.069333+0000","last_peered":"2024-09-13T16:52:04.069333+0000","last_clean":"2024-09-13T16:52:04.069333+0000","last_became_active":"2024-09-13T16:52:02.585960+0000","last_became_peered":"2024-09-13T16:52:02.585960+0000","last_unstale":"2024-09-13T16:52:04.069333+0000","last_undegraded":"2024-09-13T16:52:04.069333+0000","last_fullsized":"2024-09-13T16:52:04.069333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:32.220252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.089864+0000","last_change":"2024-09-13T16:52:03.086626+0000","last_active":"2024-09-13T16:52:03.089864+0000","last_peered":"2024-09-13T16:52:03.089864+0000","last_clean":"2024-09-13T16:52:03.089864+0000","last_became_active":"2024-09-13T16:52:03.086309+0000","last_became_peered":"2024-09-13T16:52:03.086309+0000","last_unstale":"2024-09-13T16:52:03.089864+0000","last_undegraded":"2024-09-13T16:52:03.089864+0000","last_fullsized":"2024-09-13T16:52:03.089864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.630647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068280+0000","last_change":"2024-09-13T16:52:02.580190+0000","last_active":"2024-09-13T16:52:04.068280+0000","last_peered":"2024-09-13T16:52:04.068280+0000","last_clean":"2024-09-13T16:52:04.068280+0000","last_became_active":"2024-09-13T16:52:02.579883+0000","last_became_peered":"2024-09-13T16:52:02.579883+0000","last_unstale":"2024-09-13T16:52:04.068280+0000","last_undegraded":"2024-09-13T16:52:04.068280+0000","last_fullsized":"2024-09-13T16:52:04.068280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:09.706886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.094770+0000","last_change":"2024-09-13T16:52:03.088185+0000","last_active":"2024-09-13T16:52:03.094770+0000","last_peered":"2024-09-13T16:52:03.094770+0000","last_clean":"2024-09-13T16:52:03.094770+0000","last_became_active":"2024-09-13T16:52:03.087013+0000","last_became_peered":"2024-09-13T16:52:03.087013+0000","last_unstale":"2024-09-13T16:52:03.094770+0000","last_undegraded":"2024-09-13T16:52:03.094770+0000","last_fullsized":"2024-09-13T16:52:03.094770+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:02.676020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069179+0000","last_change":"2024-09-13T16:52:02.598388+0000","last_active":"2024-09-13T16:52:04.069179+0000","last_peered":"2024-09-13T16:52:04.069179+0000","last_clean":"2024-09-13T16:52:04.069179+0000","last_became_active":"2024-09-13T16:52:02.598113+0000","last_became_peered":"2024-09-13T16:52:02.598113+0000","last_unstale":"2024-09-13T16:52:04.069179+0000","last_undegraded":"2024-09-13T16:52:04.069179+0000","last_fullsized":"2024-09-13T16:52:04.069179+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:52.951813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.085267+0000","last_change":"2024-09-13T16:52:03.083366+0000","last_active":"2024-09-13T16:52:03.085267+0000","last_peered":"2024-09-13T16:52:03.085267+0000","last_clean":"2024-09-13T16:52:03.085267+0000","last_became_active":"2024-09-13T16:52:03.083118+0000","last_became_peered":"2024-09-13T16:52:03.083118+0000","last_unstale":"2024-09-13T16:52:03.085267+0000","last_undegraded":"2024-09-13T16:52:03.085267+0000","last_fullsized":"2024-09-13T16:52:03.085267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:29.500537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069164+0000","last_change":"2024-09-13T16:52:02.589809+0000","last_active":"2024-09-13T16:52:04.069164+0000","last_peered":"2024-09-13T16:52:04.069164+0000","last_clean":"2024-09-13T16:52:04.069164+0000","last_became_active":"2024-09-13T16:52:02.589465+0000","last_became_peered":"2024-09-13T16:52:02.589465+0000","last_unstale":"2024-09-13T16:52:04.069164+0000","last_undegraded":"2024-09-13T16:52:04.069164+0000","last_fullsized":"2024-09-13T16:52:04.069164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:45.332315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069939+0000","last_change":"2024-09-13T16:52:03.085819+0000","last_active":"2024-09-13T16:52:04.069939+0000","last_peered":"2024-09-13T16:52:04.069939+0000","last_clean":"2024-09-13T16:52:04.069939+0000","last_became_active":"2024-09-13T16:52:03.085565+0000","last_became_peered":"2024-09-13T16:52:03.085565+0000","last_unstale":"2024-09-13T16:52:04.069939+0000","last_undegraded":"2024-09-13T16:52:04.069939+0000","last_fullsized":"2024-09-13T16:52:04.069939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:09.534736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068004+0000","last_change":"2024-09-13T16:52:02.596407+0000","last_active":"2024-09-13T16:52:04.068004+0000","last_peered":"2024-09-13T16:52:04.068004+0000","last_clean":"2024-09-13T16:52:04.068004+0000","last_became_active":"2024-09-13T16:52:02.596145+0000","last_became_peered":"2024-09-13T16:52:02.596145+0000","last_unstale":"2024-09-13T16:52:04.068004+0000","last_undegraded":"2024-09-13T16:52:04.068004+0000","last_fullsized":"2024-09-13T16:52:04.068004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:05.984044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.091887+0000","last_change":"2024-09-13T16:52:03.090270+0000","last_active":"2024-09-13T16:52:03.091887+0000","last_peered":"2024-09-13T16:52:03.091887+0000","last_clean":"2024-09-13T16:52:03.091887+0000","last_became_active":"2024-09-13T16:52:03.090030+0000","last_became_peered":"2024-09-13T16:52:03.090030+0000","last_unstale":"2024-09-13T16:52:03.091887+0000","last_undegraded":"2024-09-13T16:52:03.091887+0000","last_fullsized":"2024-09-13T16:52:03.091887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:58.383411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067825+0000","last_change":"2024-09-13T16:52:02.592615+0000","last_active":"2024-09-13T16:52:04.067825+0000","last_peered":"2024-09-13T16:52:04.067825+0000","last_clean":"2024-09-13T16:52:04.067825+0000","last_became_active":"2024-09-13T16:52:02.592297+0000","last_became_peered":"2024-09-13T16:52:02.592297+0000","last_unstale":"2024-09-13T16:52:04.067825+0000","last_undegraded":"2024-09-13T16:52:04.067825+0000","last_fullsized":"2024-09-13T16:52:04.067825+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:10.230749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067584+0000","last_change":"2024-09-13T16:52:03.075624+0000","last_active":"2024-09-13T16:52:04.067584+0000","last_peered":"2024-09-13T16:52:04.067584+0000","last_clean":"2024-09-13T16:52:04.067584+0000","last_became_active":"2024-09-13T16:52:03.075350+0000","last_became_peered":"2024-09-13T16:52:03.075350+0000","last_unstale":"2024-09-13T16:52:04.067584+0000","last_undegraded":"2024-09-13T16:52:04.067584+0000","last_fullsized":"2024-09-13T16:52:04.067584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:52.988622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.074553+0000","last_change":"2024-09-13T16:52:02.595054+0000","last_active":"2024-09-13T16:52:03.074553+0000","last_peered":"2024-09-13T16:52:03.074553+0000","last_clean":"2024-09-13T16:52:03.074553+0000","last_became_active":"2024-09-13T16:52:02.594705+0000","last_became_peered":"2024-09-13T16:52:02.594705+0000","last_unstale":"2024-09-13T16:52:03.074553+0000","last_undegraded":"2024-09-13T16:52:03.074553+0000","last_fullsized":"2024-09-13T16:52:03.074553+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:06.193182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.091983+0000","last_change":"2024-09-13T16:52:03.087279+0000","last_active":"2024-09-13T16:52:03.091983+0000","last_peered":"2024-09-13T16:52:03.091983+0000","last_clean":"2024-09-13T16:52:03.091983+0000","last_became_active":"2024-09-13T16:52:03.086965+0000","last_became_peered":"2024-09-13T16:52:03.086965+0000","last_unstale":"2024-09-13T16:52:03.091983+0000","last_undegraded":"2024-09-13T16:52:03.091983+0000","last_fullsized":"2024-09-13T16:52:03.091983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:15.250882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.062063+0000","last_change":"2024-09-13T16:52:02.584595+0000","last_active":"2024-09-13T16:52:03.062063+0000","last_peered":"2024-09-13T16:52:03.062063+0000","last_clean":"2024-09-13T16:52:03.062063+0000","last_became_active":"2024-09-13T16:52:02.584354+0000","last_became_peered":"2024-09-13T16:52:02.584354+0000","last_unstale":"2024-09-13T16:52:03.062063+0000","last_undegraded":"2024-09-13T16:52:03.062063+0000","last_fullsized":"2024-09-13T16:52:03.062063+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:18.283316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067613+0000","last_change":"2024-09-13T16:52:03.092913+0000","last_active":"2024-09-13T16:52:04.067613+0000","last_peered":"2024-09-13T16:52:04.067613+0000","last_clean":"2024-09-13T16:52:04.067613+0000","last_became_active":"2024-09-13T16:52:03.092656+0000","last_became_peered":"2024-09-13T16:52:03.092656+0000","last_unstale":"2024-09-13T16:52:04.067613+0000","last_undegraded":"2024-09-13T16:52:04.067613+0000","last_fullsized":"2024-09-13T16:52:04.067613+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:02:26.664729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.096718+0000","last_change":"2024-09-13T16:52:02.584744+0000","last_active":"2024-09-13T16:52:03.096718+0000","last_peered":"2024-09-13T16:52:03.096718+0000","last_clean":"2024-09-13T16:52:03.096718+0000","last_became_active":"2024-09-13T16:52:02.584454+0000","last_became_peered":"2024-09-13T16:52:02.584454+0000","last_unstale":"2024-09-13T16:52:03.096718+0000","last_undegraded":"2024-09-13T16:52:03.096718+0000","last_fullsized":"2024-09-13T16:52:03.096718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:30.321115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067896+0000","last_change":"2024-09-13T16:52:03.082946+0000","last_active":"2024-09-13T16:52:04.067896+0000","last_peered":"2024-09-13T16:52:04.067896+0000","last_clean":"2024-09-13T16:52:04.067896+0000","last_became_active":"2024-09-13T16:52:03.082687+0000","last_became_peered":"2024-09-13T16:52:03.082687+0000","last_unstale":"2024-09-13T16:52:04.067896+0000","last_undegraded":"2024-09-13T16:52:04.067896+0000","last_fullsized":"2024-09-13T16:52:04.067896+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:02.621626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069057+0000","last_change":"2024-09-13T16:52:02.589160+0000","last_active":"2024-09-13T16:52:04.069057+0000","last_peered":"2024-09-13T16:52:04.069057+0000","last_clean":"2024-09-13T16:52:04.069057+0000","last_became_active":"2024-09-13T16:52:02.588178+0000","last_became_peered":"2024-09-13T16:52:02.588178+0000","last_unstale":"2024-09-13T16:52:04.069057+0000","last_undegraded":"2024-09-13T16:52:04.069057+0000","last_fullsized":"2024-09-13T16:52:04.069057+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:32.220252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068973+0000","last_change":"2024-09-13T16:52:03.080651+0000","last_active":"2024-09-13T16:52:04.068973+0000","last_peered":"2024-09-13T16:52:04.068973+0000","last_clean":"2024-09-13T16:52:04.068973+0000","last_became_active":"2024-09-13T16:52:03.080071+0000","last_became_peered":"2024-09-13T16:52:03.080071+0000","last_unstale":"2024-09-13T16:52:04.068973+0000","last_undegraded":"2024-09-13T16:52:04.068973+0000","last_fullsized":"2024-09-13T16:52:04.068973+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:44.839062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.070031+0000","last_change":"2024-09-13T16:52:02.593248+0000","last_active":"2024-09-13T16:52:04.070031+0000","last_peered":"2024-09-13T16:52:04.070031+0000","last_clean":"2024-09-13T16:52:04.070031+0000","last_became_active":"2024-09-13T16:52:02.592722+0000","last_became_peered":"2024-09-13T16:52:02.592722+0000","last_unstale":"2024-09-13T16:52:04.070031+0000","last_undegraded":"2024-09-13T16:52:04.070031+0000","last_fullsized":"2024-09-13T16:52:04.070031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:52.951813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069811+0000","last_change":"2024-09-13T16:52:03.095454+0000","last_active":"2024-09-13T16:52:04.069811+0000","last_peered":"2024-09-13T16:52:04.069811+0000","last_clean":"2024-09-13T16:52:04.069811+0000","last_became_active":"2024-09-13T16:52:03.095191+0000","last_became_peered":"2024-09-13T16:52:03.095191+0000","last_unstale":"2024-09-13T16:52:04.069811+0000","last_undegraded":"2024-09-13T16:52:04.069811+0000","last_fullsized":"2024-09-13T16:52:04.069811+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:11:21.464886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.085295+0000","last_change":"2024-09-13T16:52:02.585970+0000","last_active":"2024-09-13T16:52:03.085295+0000","last_peered":"2024-09-13T16:52:03.085295+0000","last_clean":"2024-09-13T16:52:03.085295+0000","last_became_active":"2024-09-13T16:52:02.585718+0000","last_became_peered":"2024-09-13T16:52:02.585718+0000","last_unstale":"2024-09-13T16:52:03.085295+0000","last_undegraded":"2024-09-13T16:52:03.085295+0000","last_fullsized":"2024-09-13T16:52:03.085295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:09:01.467945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.099687+0000","last_change":"2024-09-13T16:52:03.098358+0000","last_active":"2024-09-13T16:52:03.099687+0000","last_peered":"2024-09-13T16:52:03.099687+0000","last_clean":"2024-09-13T16:52:03.099687+0000","last_became_active":"2024-09-13T16:52:03.097914+0000","last_became_peered":"2024-09-13T16:52:03.097914+0000","last_unstale":"2024-09-13T16:52:03.099687+0000","last_undegraded":"2024-09-13T16:52:03.099687+0000","last_fullsized":"2024-09-13T16:52:03.099687+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:46.496049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.075011+0000","last_change":"2024-09-13T16:52:02.587282+0000","last_active":"2024-09-13T16:52:03.075011+0000","last_peered":"2024-09-13T16:52:03.075011+0000","last_clean":"2024-09-13T16:52:03.075011+0000","last_became_active":"2024-09-13T16:52:02.587019+0000","last_became_peered":"2024-09-13T16:52:02.587019+0000","last_unstale":"2024-09-13T16:52:03.075011+0000","last_undegraded":"2024-09-13T16:52:03.075011+0000","last_fullsized":"2024-09-13T16:52:03.075011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:32.220252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.081093+0000","last_change":"2024-09-13T16:52:03.079064+0000","last_active":"2024-09-13T16:52:03.081093+0000","last_peered":"2024-09-13T16:52:03.081093+0000","last_clean":"2024-09-13T16:52:03.081093+0000","last_became_active":"2024-09-13T16:52:03.078804+0000","last_became_peered":"2024-09-13T16:52:03.078804+0000","last_unstale":"2024-09-13T16:52:03.081093+0000","last_undegraded":"2024-09-13T16:52:03.081093+0000","last_fullsized":"2024-09-13T16:52:03.081093+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:46.496049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.084516+0000","last_change":"2024-09-13T16:52:02.591479+0000","last_active":"2024-09-13T16:52:03.084516+0000","last_peered":"2024-09-13T16:52:03.084516+0000","last_clean":"2024-09-13T16:52:03.084516+0000","last_became_active":"2024-09-13T16:52:02.591244+0000","last_became_peered":"2024-09-13T16:52:02.591244+0000","last_unstale":"2024-09-13T16:52:03.084516+0000","last_undegraded":"2024-09-13T16:52:03.084516+0000","last_fullsized":"2024-09-13T16:52:03.084516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:30.321115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.083858+0000","last_change":"2024-09-13T16:52:03.080594+0000","last_active":"2024-09-13T16:52:03.083858+0000","last_peered":"2024-09-13T16:52:03.083858+0000","last_clean":"2024-09-13T16:52:03.083858+0000","last_became_active":"2024-09-13T16:52:03.080337+0000","last_became_peered":"2024-09-13T16:52:03.080337+0000","last_unstale":"2024-09-13T16:52:03.083858+0000","last_undegraded":"2024-09-13T16:52:03.083858+0000","last_fullsized":"2024-09-13T16:52:03.083858+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:14.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.068976+0000","last_change":"2024-09-13T16:52:02.586883+0000","last_active":"2024-09-13T16:52:03.068976+0000","last_peered":"2024-09-13T16:52:03.068976+0000","last_clean":"2024-09-13T16:52:03.068976+0000","last_became_active":"2024-09-13T16:52:02.586370+0000","last_became_peered":"2024-09-13T16:52:02.586370+0000","last_unstale":"2024-09-13T16:52:03.068976+0000","last_undegraded":"2024-09-13T16:52:03.068976+0000","last_fullsized":"2024-09-13T16:52:03.068976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:10.154122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068281+0000","last_change":"2024-09-13T16:52:03.067442+0000","last_active":"2024-09-13T16:52:04.068281+0000","last_peered":"2024-09-13T16:52:04.068281+0000","last_clean":"2024-09-13T16:52:04.068281+0000","last_became_active":"2024-09-13T16:52:03.066943+0000","last_became_peered":"2024-09-13T16:52:03.066943+0000","last_unstale":"2024-09-13T16:52:04.068281+0000","last_undegraded":"2024-09-13T16:52:04.068281+0000","last_fullsized":"2024-09-13T16:52:04.068281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:11:21.464886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.067292+0000","last_change":"2024-09-13T16:52:02.583895+0000","last_active":"2024-09-13T16:52:03.067292+0000","last_peered":"2024-09-13T16:52:03.067292+0000","last_clean":"2024-09-13T16:52:03.067292+0000","last_became_active":"2024-09-13T16:52:02.583629+0000","last_became_peered":"2024-09-13T16:52:02.583629+0000","last_unstale":"2024-09-13T16:52:03.067292+0000","last_undegraded":"2024-09-13T16:52:03.067292+0000","last_fullsized":"2024-09-13T16:52:03.067292+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:05.265636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067712+0000","last_change":"2024-09-13T16:52:03.093629+0000","last_active":"2024-09-13T16:52:04.067712+0000","last_peered":"2024-09-13T16:52:04.067712+0000","last_clean":"2024-09-13T16:52:04.067712+0000","last_became_active":"2024-09-13T16:52:03.081106+0000","last_became_peered":"2024-09-13T16:52:03.081106+0000","last_unstale":"2024-09-13T16:52:04.067712+0000","last_undegraded":"2024-09-13T16:52:04.067712+0000","last_fullsized":"2024-09-13T16:52:04.067712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:52.450693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.068821+0000","last_change":"2024-09-13T16:52:02.584869+0000","last_active":"2024-09-13T16:52:03.068821+0000","last_peered":"2024-09-13T16:52:03.068821+0000","last_clean":"2024-09-13T16:52:03.068821+0000","last_became_active":"2024-09-13T16:52:02.584582+0000","last_became_peered":"2024-09-13T16:52:02.584582+0000","last_unstale":"2024-09-13T16:52:03.068821+0000","last_undegraded":"2024-09-13T16:52:03.068821+0000","last_fullsized":"2024-09-13T16:52:03.068821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:53.118849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.091090+0000","last_change":"2024-09-13T16:52:03.082736+0000","last_active":"2024-09-13T16:52:03.091090+0000","last_peered":"2024-09-13T16:52:03.091090+0000","last_clean":"2024-09-13T16:52:03.091090+0000","last_became_active":"2024-09-13T16:52:03.082456+0000","last_became_peered":"2024-09-13T16:52:03.082456+0000","last_unstale":"2024-09-13T16:52:03.091090+0000","last_undegraded":"2024-09-13T16:52:03.091090+0000","last_fullsized":"2024-09-13T16:52:03.091090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:06.994719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069132+0000","last_change":"2024-09-13T16:52:02.593461+0000","last_active":"2024-09-13T16:52:04.069132+0000","last_peered":"2024-09-13T16:52:04.069132+0000","last_clean":"2024-09-13T16:52:04.069132+0000","last_became_active":"2024-09-13T16:52:02.593156+0000","last_became_peered":"2024-09-13T16:52:02.593156+0000","last_unstale":"2024-09-13T16:52:04.069132+0000","last_undegraded":"2024-09-13T16:52:04.069132+0000","last_fullsized":"2024-09-13T16:52:04.069132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:06.193182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.095332+0000","last_change":"2024-09-13T16:52:03.089031+0000","last_active":"2024-09-13T16:52:03.095332+0000","last_peered":"2024-09-13T16:52:03.095332+0000","last_clean":"2024-09-13T16:52:03.095332+0000","last_became_active":"2024-09-13T16:52:03.088746+0000","last_became_peered":"2024-09-13T16:52:03.088746+0000","last_unstale":"2024-09-13T16:52:03.095332+0000","last_undegraded":"2024-09-13T16:52:03.095332+0000","last_fullsized":"2024-09-13T16:52:03.095332+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:32.675438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.077561+0000","last_change":"2024-09-13T16:52:02.588087+0000","last_active":"2024-09-13T16:52:03.077561+0000","last_peered":"2024-09-13T16:52:03.077561+0000","last_clean":"2024-09-13T16:52:03.077561+0000","last_became_active":"2024-09-13T16:52:02.587804+0000","last_became_peered":"2024-09-13T16:52:02.587804+0000","last_unstale":"2024-09-13T16:52:03.077561+0000","last_undegraded":"2024-09-13T16:52:03.077561+0000","last_fullsized":"2024-09-13T16:52:03.077561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:51.707645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.068139+0000","last_change":"2024-09-13T16:52:03.100910+0000","last_active":"2024-09-13T16:52:04.068139+0000","last_peered":"2024-09-13T16:52:04.068139+0000","last_clean":"2024-09-13T16:52:04.068139+0000","last_became_active":"2024-09-13T16:52:03.098876+0000","last_became_peered":"2024-09-13T16:52:03.098876+0000","last_unstale":"2024-09-13T16:52:04.068139+0000","last_undegraded":"2024-09-13T16:52:04.068139+0000","last_fullsized":"2024-09-13T16:52:04.068139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:06.544382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.073901+0000","last_change":"2024-09-13T16:52:02.590912+0000","last_active":"2024-09-13T16:52:03.073901+0000","last_peered":"2024-09-13T16:52:03.073901+0000","last_clean":"2024-09-13T16:52:03.073901+0000","last_became_active":"2024-09-13T16:52:02.590498+0000","last_became_peered":"2024-09-13T16:52:02.590498+0000","last_unstale":"2024-09-13T16:52:03.073901+0000","last_undegraded":"2024-09-13T16:52:03.073901+0000","last_fullsized":"2024-09-13T16:52:03.073901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:18.283316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.101443+0000","last_change":"2024-09-13T16:52:03.098763+0000","last_active":"2024-09-13T16:52:03.101443+0000","last_peered":"2024-09-13T16:52:03.101443+0000","last_clean":"2024-09-13T16:52:03.101443+0000","last_became_active":"2024-09-13T16:52:03.098460+0000","last_became_peered":"2024-09-13T16:52:03.098460+0000","last_unstale":"2024-09-13T16:52:03.101443+0000","last_undegraded":"2024-09-13T16:52:03.101443+0000","last_fullsized":"2024-09-13T16:52:03.101443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:27:40.756415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.077710+0000","last_change":"2024-09-13T16:52:02.590814+0000","last_active":"2024-09-13T16:52:03.077710+0000","last_peered":"2024-09-13T16:52:03.077710+0000","last_clean":"2024-09-13T16:52:03.077710+0000","last_became_active":"2024-09-13T16:52:02.590540+0000","last_became_peered":"2024-09-13T16:52:02.590540+0000","last_unstale":"2024-09-13T16:52:03.077710+0000","last_undegraded":"2024-09-13T16:52:03.077710+0000","last_fullsized":"2024-09-13T16:52:03.077710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:45.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.086177+0000","last_change":"2024-09-13T16:52:03.074662+0000","last_active":"2024-09-13T16:52:03.086177+0000","last_peered":"2024-09-13T16:52:03.086177+0000","last_clean":"2024-09-13T16:52:03.086177+0000","last_became_active":"2024-09-13T16:52:03.074189+0000","last_became_peered":"2024-09-13T16:52:03.074189+0000","last_unstale":"2024-09-13T16:52:03.086177+0000","last_undegraded":"2024-09-13T16:52:03.086177+0000","last_fullsized":"2024-09-13T16:52:03.086177+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:40.831264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.064223+0000","last_change":"2024-09-13T16:52:02.585055+0000","last_active":"2024-09-13T16:52:03.064223+0000","last_peered":"2024-09-13T16:52:03.064223+0000","last_clean":"2024-09-13T16:52:03.064223+0000","last_became_active":"2024-09-13T16:52:02.583595+0000","last_became_peered":"2024-09-13T16:52:02.583595+0000","last_unstale":"2024-09-13T16:52:03.064223+0000","last_undegraded":"2024-09-13T16:52:03.064223+0000","last_fullsized":"2024-09-13T16:52:03.064223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:45.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069567+0000","last_change":"2024-09-13T16:52:03.098609+0000","last_active":"2024-09-13T16:52:04.069567+0000","last_peered":"2024-09-13T16:52:04.069567+0000","last_clean":"2024-09-13T16:52:04.069567+0000","last_became_active":"2024-09-13T16:52:03.098374+0000","last_became_peered":"2024-09-13T16:52:03.098374+0000","last_unstale":"2024-09-13T16:52:04.069567+0000","last_undegraded":"2024-09-13T16:52:04.069567+0000","last_fullsized":"2024-09-13T16:52:04.069567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:30:06.994719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069738+0000","last_change":"2024-09-13T16:52:02.590767+0000","last_active":"2024-09-13T16:52:04.069738+0000","last_peered":"2024-09-13T16:52:04.069738+0000","last_clean":"2024-09-13T16:52:04.069738+0000","last_became_active":"2024-09-13T16:52:02.590437+0000","last_became_peered":"2024-09-13T16:52:02.590437+0000","last_unstale":"2024-09-13T16:52:04.069738+0000","last_undegraded":"2024-09-13T16:52:04.069738+0000","last_fullsized":"2024-09-13T16:52:04.069738+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:06.193182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.092113+0000","last_change":"2024-09-13T16:52:03.084537+0000","last_active":"2024-09-13T16:52:03.092113+0000","last_peered":"2024-09-13T16:52:03.092113+0000","last_clean":"2024-09-13T16:52:03.092113+0000","last_became_active":"2024-09-13T16:52:03.084187+0000","last_became_peered":"2024-09-13T16:52:03.084187+0000","last_unstale":"2024-09-13T16:52:03.092113+0000","last_undegraded":"2024-09-13T16:52:03.092113+0000","last_fullsized":"2024-09-13T16:52:03.092113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:41.160361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.067596+0000","last_change":"2024-09-13T16:52:02.596350+0000","last_active":"2024-09-13T16:52:04.067596+0000","last_peered":"2024-09-13T16:52:04.067596+0000","last_clean":"2024-09-13T16:52:04.067596+0000","last_became_active":"2024-09-13T16:52:02.596087+0000","last_became_peered":"2024-09-13T16:52:02.596087+0000","last_unstale":"2024-09-13T16:52:04.067596+0000","last_undegraded":"2024-09-13T16:52:04.067596+0000","last_fullsized":"2024-09-13T16:52:04.067596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:08.524061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T16:52:04.069001+0000","last_change":"2024-09-13T16:52:03.083118+0000","last_active":"2024-09-13T16:52:04.069001+0000","last_peered":"2024-09-13T16:52:04.069001+0000","last_clean":"2024-09-13T16:52:04.069001+0000","last_became_active":"2024-09-13T16:52:03.082858+0000","last_became_peered":"2024-09-13T16:52:03.082858+0000","last_unstale":"2024-09-13T16:52:04.069001+0000","last_undegraded":"2024-09-13T16:52:04.069001+0000","last_fullsized":"2024-09-13T16:52:04.069001+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:02.553469+0000","last_clean_scrub_stamp":"2024-09-13T16:52:02.553469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:19.241815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T16:52:03.068848+0000","last_change":"2024-09-13T16:52:02.586740+0000","last_active":"2024-09-13T16:52:03.068848+0000","last_peered":"2024-09-13T16:52:03.068848+0000","last_clean":"2024-09-13T16:52:03.068848+0000","last_became_active":"2024-09-13T16:52:02.586493+0000","last_became_peered":"2024-09-13T16:52:02.586493+0000","last_unstale":"2024-09-13T16:52:03.068848+0000","last_undegraded":"2024-09-13T16:52:03.068848+0000","last_fullsized":"2024-09-13T16:52:03.068848+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:01.542794+0000","last_clean_scrub_stamp":"2024-09-13T16:52:01.542794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:10.154122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T16:52:02.571731+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:52:02.571731+0000","last_peered":"2024-09-13T16:52:02.571731+0000","last_clean":"2024-09-13T16:52:02.571731+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:52:02.571731+0000","last_undegraded":"2024-09-13T16:52:02.571731+0000","last_fullsized":"2024-09-13T16:52:02.571731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1060864,"data_stored":786562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.466}]},{"osd":1,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.083},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.5920000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":5.6959999999999997,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":5.6260000000000003,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.544}]},{"osd":3,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.663}]},{"osd":4,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.581},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.6140000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":3.0139999999999998},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":2.7869999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.5660000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":454656,"data_stored":195142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":3.1739999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.999}]},{"osd":1,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.3320000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.139}]},{"osd":3,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":1.137,"5min":1.137,"15min":1.137},"max":{"1min":3.3769999999999998,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.4950000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.272},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.377}]},{"osd":5,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.6789999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":1.232},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.1779999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":3.2250000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.417}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":458752,"data_stored":194672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.175},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.2}]},{"osd":1,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":1.026,"5min":1.026,"15min":1.026},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.23}]},{"osd":2,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.405}]},{"osd":3,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.8300000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.0780000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.304},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.534}]},{"osd":6,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.4990000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":3.5630000000000002,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.2809999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":471040,"data_stored":201784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.3280000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.411},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.6080000000000001,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.5169999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":4.8099999999999996,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.125},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.204},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.268}]},{"osd":7,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.579,"15min":1.579},"min":{"1min":1.107,"5min":1.107,"15min":1.107},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":2.6619999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.468}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":196158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.175}]},{"osd":1,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.129}]},{"osd":4,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.339},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.23}]},{"osd":5,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.397}]},{"osd":6,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.466}]},{"osd":7,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.3600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.331}]},{"osd":1,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.012}]},{"osd":3,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.137}]},{"osd":4,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.1930000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.5469999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"min":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":1.073,"5min":1.073,"15min":1.073},"max":{"1min":3.3980000000000001,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.637},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"max":{"1min":3.4060000000000001,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.2589999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0580000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.7610000000000001,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.345},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.9630000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.0189999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":2.7440000000000002,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.2629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":784962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.0149999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.5430000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.147}]},{"osd":4,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.1879999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":3.3999999999999999,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":1.383},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.9860000000000002,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.696}]},{"osd":6,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"max":{"1min":3.3039999999999998,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":1.466}]},{"osd":7,"last update":"Fri Sep 13 16:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.7789999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:52:07.920 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T16:52:07.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T16:52:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.156+0000 7f4571400700 1 Processor -- start 2024-09-13T16:52:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.156+0000 7f4571400700 1 -- start start 2024-09-13T16:52:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f4571400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f456c15c990 0x7f456c0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f4571400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f456c0cd390 0x7f456c0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f4571400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0cc0 0x7f456c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f4571400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c0b2230 con 0x7f456c0cd390 2024-09-13T16:52:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f4571400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c0b1190 con 0x7f456c15c990 2024-09-13T16:52:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f4571400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0b1470 con 0x7f456c0b0cc0 2024-09-13T16:52:08.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456b400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f456c0cd390 0x7f456c0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456b400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f456c0cd390 0x7f456c0b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44956/0 (socket says 172.21.15.3:44956) 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456b400700 1 -- 172.21.15.3:0/3186197009 learned_addr learned my addr 172.21.15.3:0/3186197009 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456be00700 1 --2- 172.21.15.3:0/3186197009 >> v2:172.21.15.157:3300/0 conn(0x7f456c15c990 0x7f456c0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f4570a00700 1 --2- 172.21.15.3:0/3186197009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0cc0 0x7f456c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456b400700 1 -- 172.21.15.3:0/3186197009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0cc0 msgr2=0x7f456c0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456b400700 1 --2- 172.21.15.3:0/3186197009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0cc0 0x7f456c0b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456b400700 1 -- 172.21.15.3:0/3186197009 >> v2:172.21.15.157:3300/0 conn(0x7f456c15c990 msgr2=0x7f456c0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456b400700 1 --2- 172.21.15.3:0/3186197009 >> v2:172.21.15.157:3300/0 conn(0x7f456c15c990 0x7f456c0b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f456b400700 1 -- 172.21.15.3:0/3186197009 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0b1750 con 0x7f456c0cd390 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.160+0000 7f4570a00700 1 --2- 172.21.15.3:0/3186197009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0cc0 0x7f456c0b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f456b400700 1 --2- 172.21.15.3:0/3186197009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0cd390 0x7f456c0b4970 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f456c0b23f0 tx=0x7f4558009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40533c5130281f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:08.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f456aa00700 1 -- 172.21.15.3:0/3186197009 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800b8a0 con 0x7f456c0cd390 2024-09-13T16:52:08.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f456aa00700 1 -- 172.21.15.3:0/3186197009 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455800ba00 con 0x7f456c0cd390 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f456aa00700 1 -- 172.21.15.3:0/3186197009 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800bd90 con 0x7f456c0cd390 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 -- 172.21.15.3:0/3186197009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0cd390 msgr2=0x7f456c0b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 --2- 172.21.15.3:0/3186197009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0cd390 0x7f456c0b4970 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f456c0b23f0 tx=0x7f4558009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 -- 172.21.15.3:0/3186197009 shutdown_connections 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 --2- 172.21.15.3:0/3186197009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0cd390 0x7f456c0b4970 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 --2- 172.21.15.3:0/3186197009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0cc0 0x7f456c0b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 --2- 172.21.15.3:0/3186197009 >> v2:172.21.15.157:3300/0 conn(0x7f456c15c990 0x7f456c0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 -- 172.21.15.3:0/3186197009 >> 172.21.15.3:0/3186197009 conn(0x7f456c0721a0 msgr2=0x7f456c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 -- 172.21.15.3:0/3186197009 shutdown_connections 2024-09-13T16:52:08.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.164+0000 7f4571400700 1 -- 172.21.15.3:0/3186197009 wait complete. 2024-09-13T16:52:08.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 Processor -- start 2024-09-13T16:52:08.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 -- start start 2024-09-13T16:52:08.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0b0cc0 0x7f456c148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0cd390 0x7f456c14d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c15c990 0x7f456c14dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c06dbb0 con 0x7f456c0b0cc0 2024-09-13T16:52:08.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c06de90 con 0x7f456c0cd390 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c06e170 con 0x7f456c15c990 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0b0cc0 0x7f456c148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0b0cc0 0x7f456c148430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44972/0 (socket says 172.21.15.3:44972) 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 -- 172.21.15.3:0/3306856157 learned_addr learned my addr 172.21.15.3:0/3306856157 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4570a00700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c15c990 0x7f456c14dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456b400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0cd390 0x7f456c14d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 -- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c15c990 msgr2=0x7f456c14dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c15c990 0x7f456c14dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 -- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0cd390 msgr2=0x7f456c14d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0cd390 0x7f456c14d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 -- 172.21.15.3:0/3306856157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558009570 con 0x7f456c0b0cc0 2024-09-13T16:52:08.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f456be00700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0b0cc0 0x7f456c148430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f455c00d800 tx=0x7f455c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:08.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4563e00700 1 -- 172.21.15.3:0/3306856157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c0125f0 con 0x7f456c0b0cc0 2024-09-13T16:52:08.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4563e00700 1 -- 172.21.15.3:0/3306856157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c012750 con 0x7f456c0b0cc0 2024-09-13T16:52:08.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c06e450 con 0x7f456c0b0cc0 2024-09-13T16:52:08.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4563e00700 1 -- 172.21.15.3:0/3306856157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c012ac0 con 0x7f456c0b0cc0 2024-09-13T16:52:08.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.168+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c148970 con 0x7f456c0b0cc0 2024-09-13T16:52:08.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.172+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c0cb850 con 0x7f456c0b0cc0 2024-09-13T16:52:08.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.172+0000 7f4563e00700 1 -- 172.21.15.3:0/3306856157 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f455c012c20 con 0x7f456c0b0cc0 2024-09-13T16:52:08.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.176+0000 7f4563e00700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f454c066730 0x7f454c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.176+0000 7f4563e00700 1 -- 172.21.15.3:0/3306856157 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f455c084690 con 0x7f456c0b0cc0 2024-09-13T16:52:08.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.176+0000 7f456b400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f454c066730 0x7f454c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.176+0000 7f456b400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f454c066730 0x7f454c068bf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f456c0b23f0 tx=0x7f4558008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:08.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.180+0000 7f4563e00700 1 -- 172.21.15.3:0/3306856157 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f456c0cb850 con 0x7f456c0b0cc0 2024-09-13T16:52:08.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.432+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f456c0cb850 con 0x7f456c0b0cc0 2024-09-13T16:52:08.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.436+0000 7f4563e00700 1 -- 172.21.15.3:0/3306856157 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f456c0cb850 con 0x7f456c0b0cc0 2024-09-13T16:52:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f454c066730 msgr2=0x7f454c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f454c066730 0x7f454c068bf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f456c0b23f0 tx=0x7f4558008030 comp rx=0 tx=0).stop 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0b0cc0 msgr2=0x7f456c148430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0b0cc0 0x7f456c148430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f455c00d800 tx=0x7f455c00dbc0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 shutdown_connections 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f454c066730 0x7f454c068bf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c15c990 0x7f456c14dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0cd390 0x7f456c14d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 --2- 172.21.15.3:0/3306856157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456c0b0cc0 0x7f456c148430 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.440+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 >> 172.21.15.3:0/3306856157 conn(0x7f456c0721a0 msgr2=0x7f456c158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.444+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 shutdown_connections 2024-09-13T16:52:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.444+0000 7f4571400700 1 -- 172.21.15.3:0/3306856157 wait complete. 2024-09-13T16:52:08.450 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-13T16:52:08.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:08.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.640+0000 7f23f0e00700 1 Processor -- start 2024-09-13T16:52:08.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.640+0000 7f23f0e00700 1 -- start start 2024-09-13T16:52:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23f0e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f23ec15cdd0 0x7f23ec0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23f0e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec0d1510 con 0x7f23ec0d25b0 2024-09-13T16:52:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23f0e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec0d17f0 con 0x7f23ec15d1b0 2024-09-13T16:52:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23f0e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ec0d1ad0 con 0x7f23ec15cdd0 2024-09-13T16:52:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23ebe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f23ec0d25b0 0x7f23ec0cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45276/0 (socket says 172.21.15.3:45276) 2024-09-13T16:52:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23eb400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f23ec15cdd0 0x7f23ec0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23ebe00700 1 -- 172.21.15.3:0/1091035185 learned_addr learned my addr 172.21.15.3:0/1091035185 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:08.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23eaa00700 1 -- 172.21.15.3:0/1091035185 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236466522 0 0) 0x7f23ec0d1510 con 0x7f23ec0d25b0 2024-09-13T16:52:08.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23eaa00700 1 -- 172.21.15.3:0/1091035185 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23d00035d0 con 0x7f23ec0d25b0 2024-09-13T16:52:08.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23eb400700 1 -- 172.21.15.3:0/1091035185 >> v1:172.21.15.157:6789/0 conn(0x7f23ec15d1b0 legacy=0x7f23ec0b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23eaa00700 1 -- 172.21.15.3:0/1091035185 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074317073 0 0) 0x7f23ec0d17f0 con 0x7f23ec15d1b0 2024-09-13T16:52:08.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23eb400700 1 -- 172.21.15.3:0/1091035185 >> v1:172.21.15.3:6789/0 conn(0x7f23ec0d25b0 legacy=0x7f23ec0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.644+0000 7f23eb400700 1 -- 172.21.15.3:0/1091035185 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec0d1db0 con 0x7f23ec15cdd0 2024-09-13T16:52:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23eb400700 1 --2- 172.21.15.3:0/1091035185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15cdd0 0x7f23ec0d0e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009850 tx=0x7f23dc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc4090ce14d0663 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23eaa00700 1 -- 172.21.15.3:0/1091035185 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc011830 con 0x7f23ec15cdd0 2024-09-13T16:52:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23eaa00700 1 -- 172.21.15.3:0/1091035185 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc011990 con 0x7f23ec15cdd0 2024-09-13T16:52:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23eaa00700 1 -- 172.21.15.3:0/1091035185 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc011d40 con 0x7f23ec15cdd0 2024-09-13T16:52:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23f0e00700 1 -- 172.21.15.3:0/1091035185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15cdd0 msgr2=0x7f23ec0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23f0e00700 1 --2- 172.21.15.3:0/1091035185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15cdd0 0x7f23ec0d0e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009850 tx=0x7f23dc009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23f0e00700 1 -- 172.21.15.3:0/1091035185 shutdown_connections 2024-09-13T16:52:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23f0e00700 1 --2- 172.21.15.3:0/1091035185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15cdd0 0x7f23ec0d0e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23f0e00700 1 -- 172.21.15.3:0/1091035185 >> 172.21.15.3:0/1091035185 conn(0x7f23ec0d3190 msgr2=0x7f23ec0d35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23f0e00700 1 -- 172.21.15.3:0/1091035185 shutdown_connections 2024-09-13T16:52:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.648+0000 7f23f0e00700 1 -- 172.21.15.3:0/1091035185 wait complete. 2024-09-13T16:52:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.652+0000 7f23f0e00700 1 Processor -- start 2024-09-13T16:52:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.652+0000 7f23f0e00700 1 -- start start 2024-09-13T16:52:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.652+0000 7f23f0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23ec0d25b0 0x7f23ec0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.652+0000 7f23f0e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23ec15cdd0 0x7f23ec14e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.652+0000 7f23f0e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15d1b0 0x7f23ec0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.652+0000 7f23f0e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0cbea0 con 0x7f23ec0d25b0 2024-09-13T16:52:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.652+0000 7f23f0e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec14ac60 con 0x7f23ec15d1b0 2024-09-13T16:52:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.652+0000 7f23f0e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0cb3d0 con 0x7f23ec15cdd0 2024-09-13T16:52:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23ec0d25b0 0x7f23ec0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23ec0d25b0 0x7f23ec0cae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44986/0 (socket says 172.21.15.3:44986) 2024-09-13T16:52:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 -- 172.21.15.3:0/4148915781 learned_addr learned my addr 172.21.15.3:0/4148915781 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23e2a00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23ec15cdd0 0x7f23ec14e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23ebe00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15d1b0 0x7f23ec0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 -- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23ec15cdd0 msgr2=0x7f23ec14e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23ec15cdd0 0x7f23ec14e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 -- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15d1b0 msgr2=0x7f23ec0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15d1b0 0x7f23ec0b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 -- 172.21.15.3:0/4148915781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc0094b0 con 0x7f23ec0d25b0 2024-09-13T16:52:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23e2a00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23ec15cdd0 0x7f23ec14e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23eb400700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23ec0d25b0 0x7f23ec0cae90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23ec067e10 tx=0x7f23dc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23e3e00700 1 -- 172.21.15.3:0/4148915781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc01ab70 con 0x7f23ec0d25b0 2024-09-13T16:52:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23e3e00700 1 -- 172.21.15.3:0/4148915781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc01f070 con 0x7f23ec0d25b0 2024-09-13T16:52:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23e3e00700 1 -- 172.21.15.3:0/4148915781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc01b430 con 0x7f23ec0d25b0 2024-09-13T16:52:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec0cb600 con 0x7f23ec0d25b0 2024-09-13T16:52:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.656+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec0cbbc0 con 0x7f23ec0d25b0 2024-09-13T16:52:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.660+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec0d1290 con 0x7f23ec0d25b0 2024-09-13T16:52:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.660+0000 7f23e3e00700 1 -- 172.21.15.3:0/4148915781 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f23dc01b590 con 0x7f23ec0d25b0 2024-09-13T16:52:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.660+0000 7f23e3e00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f23c8066730 0x7f23c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.660+0000 7f23e3e00700 1 -- 172.21.15.3:0/4148915781 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f23dc085450 con 0x7f23ec0d25b0 2024-09-13T16:52:08.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.664+0000 7f23e2a00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f23c8066730 0x7f23c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:08.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.664+0000 7f23e2a00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f23c8066730 0x7f23c8068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f23ec157300 tx=0x7f23d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:08.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.668+0000 7f23e3e00700 1 -- 172.21.15.3:0/4148915781 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f23dc052670 con 0x7f23ec0d25b0 2024-09-13T16:52:08.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.892+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23ec0d0e10 con 0x7f23ec0d25b0 2024-09-13T16:52:08.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.892+0000 7f23e3e00700 1 -- 172.21.15.3:0/4148915781 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f23dc021090 con 0x7f23ec0d25b0 2024-09-13T16:52:08.899 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:08.899 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1649229219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1649229219},{"type":"v1","addr":"172.21.15.3:6833","nonce":1649229219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3459985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3459985282},{"type":"v1","addr":"172.21.15.157:6835","nonce":3459985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3779761732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3779761732},{"type":"v1","addr":"172.21.15.157:6837","nonce":3779761732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:06.068076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5373":{"gid":5373,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/750657669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":750657669},{"type":"v1","addr":"172.21.15.3:6835","nonce":750657669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f23c8066730 msgr2=0x7f23c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f23c8066730 0x7f23c8068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f23ec157300 tx=0x7f23d4009040 comp rx=0 tx=0).stop 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23ec0d25b0 msgr2=0x7f23ec0cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23ec0d25b0 0x7f23ec0cae90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23ec067e10 tx=0x7f23dc004770 comp rx=0 tx=0).stop 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 shutdown_connections 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f23c8066730 0x7f23c8068bf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23ec15d1b0 0x7f23ec0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23ec15cdd0 0x7f23ec14e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 --2- 172.21.15.3:0/4148915781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23ec0d25b0 0x7f23ec0cae90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 >> 172.21.15.3:0/4148915781 conn(0x7f23ec0d3190 msgr2=0x7f23ec06bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:08.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 shutdown_connections 2024-09-13T16:52:08.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:08.896+0000 7f23f0e00700 1 -- 172.21.15.3:0/4148915781 wait complete. 2024-09-13T16:52:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 16 2024-09-13T16:52:09.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:52:03.049167+0000', 'modified': '2024-09-13T16:52:06.068076+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5373': {'gid': 5373, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.3:6835/750657669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 750657669}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 750657669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:52:09.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:52:09.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T16:52:09.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a5e600700 1 Processor -- start 2024-09-13T16:52:09.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a5e600700 1 -- start start 2024-09-13T16:52:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a5e600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a5806d0e0 0x7f1a5806d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a5e600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a580710b0 0x7f1a580714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a5e600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a580b2c90 con 0x7f1a580ce3b0 2024-09-13T16:52:09.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a5e600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5806da60 con 0x7f1a580710b0 2024-09-13T16:52:09.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a5e600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580ce790 con 0x7f1a5806d0e0 2024-09-13T16:52:09.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a57400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1a580ce3b0 0x7f1a5815dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45278/0 (socket says 172.21.15.3:45278) 2024-09-13T16:52:09.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.500+0000 7f1a57400700 1 -- 172.21.15.3:0/2715923568 learned_addr learned my addr 172.21.15.3:0/2715923568 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:09.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a57e00700 1 --2- 172.21.15.3:0/2715923568 >> v2:172.21.15.157:3300/0 conn(0x7f1a580710b0 0x7f1a580714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56a00700 1 --2- 172.21.15.3:0/2715923568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a5806d0e0 0x7f1a5806d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56000700 1 -- 172.21.15.3:0/2715923568 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742113386 0 0) 0x7f1a580b2c90 con 0x7f1a580ce3b0 2024-09-13T16:52:09.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56000700 1 -- 172.21.15.3:0/2715923568 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a3c0035d0 con 0x7f1a580ce3b0 2024-09-13T16:52:09.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56a00700 1 -- 172.21.15.3:0/2715923568 >> v2:172.21.15.157:3300/0 conn(0x7f1a580710b0 msgr2=0x7f1a580714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56a00700 1 --2- 172.21.15.3:0/2715923568 >> v2:172.21.15.157:3300/0 conn(0x7f1a580710b0 0x7f1a580714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56a00700 1 -- 172.21.15.3:0/2715923568 >> v1:172.21.15.3:6789/0 conn(0x7f1a580ce3b0 legacy=0x7f1a5815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56a00700 1 -- 172.21.15.3:0/2715923568 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a580b2080 con 0x7f1a5806d0e0 2024-09-13T16:52:09.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56a00700 1 --2- 172.21.15.3:0/2715923568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a5806d0e0 0x7f1a5806d520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009850 tx=0x7f1a44009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f81ebbc9768e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56000700 1 -- 172.21.15.3:0/2715923568 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44011830 con 0x7f1a5806d0e0 2024-09-13T16:52:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56000700 1 -- 172.21.15.3:0/2715923568 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44011990 con 0x7f1a5806d0e0 2024-09-13T16:52:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a56000700 1 -- 172.21.15.3:0/2715923568 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44011d00 con 0x7f1a5806d0e0 2024-09-13T16:52:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a5e600700 1 -- 172.21.15.3:0/2715923568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a5806d0e0 msgr2=0x7f1a5806d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2715923568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a5806d0e0 0x7f1a5806d520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009850 tx=0x7f1a44009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a5e600700 1 -- 172.21.15.3:0/2715923568 shutdown_connections 2024-09-13T16:52:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2715923568 >> v2:172.21.15.157:3300/0 conn(0x7f1a580710b0 0x7f1a580714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2715923568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a5806d0e0 0x7f1a5806d520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a5e600700 1 -- 172.21.15.3:0/2715923568 >> 172.21.15.3:0/2715923568 conn(0x7f1a5806a390 msgr2=0x7f1a5806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a5e600700 1 -- 172.21.15.3:0/2715923568 shutdown_connections 2024-09-13T16:52:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.504+0000 7f1a5e600700 1 -- 172.21.15.3:0/2715923568 wait complete. 2024-09-13T16:52:09.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a5e600700 1 Processor -- start 2024-09-13T16:52:09.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a5e600700 1 -- start start 2024-09-13T16:52:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a5e600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a5806d0e0 0x7f1a580c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a5e600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580710b0 0x7f1a581486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a5e600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a580ce3b0 0x7f1a5815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a5e600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580c3de0 con 0x7f1a580ce3b0 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a5e600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a581f3a50 con 0x7f1a5806d0e0 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a5e600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a581f2e50 con 0x7f1a580710b0 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a57e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a580ce3b0 0x7f1a5815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a57e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a580ce3b0 0x7f1a5815a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44990/0 (socket says 172.21.15.3:44990) 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.508+0000 7f1a57e00700 1 -- 172.21.15.3:0/2982657798 learned_addr learned my addr 172.21.15.3:0/2982657798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a57400700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a5806d0e0 0x7f1a580c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a56a00700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580710b0 0x7f1a581486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a57e00700 1 -- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580710b0 msgr2=0x7f1a581486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a57e00700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580710b0 0x7f1a581486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a57e00700 1 -- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a5806d0e0 msgr2=0x7f1a580c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a57e00700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a5806d0e0 0x7f1a580c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a57e00700 1 -- 172.21.15.3:0/2982657798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440094b0 con 0x7f1a580ce3b0 2024-09-13T16:52:09.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a57e00700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a580ce3b0 0x7f1a5815a9a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009d20 tx=0x7f1a5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a4f400700 1 -- 172.21.15.3:0/2982657798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000cc90 con 0x7f1a580ce3b0 2024-09-13T16:52:09.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a4f400700 1 -- 172.21.15.3:0/2982657798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5000f040 con 0x7f1a580ce3b0 2024-09-13T16:52:09.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a5e600700 1 -- 172.21.15.3:0/2982657798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a581f3130 con 0x7f1a580ce3b0 2024-09-13T16:52:09.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a4f400700 1 -- 172.21.15.3:0/2982657798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50009b10 con 0x7f1a580ce3b0 2024-09-13T16:52:09.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.512+0000 7f1a5e600700 1 -- 172.21.15.3:0/2982657798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a581f36f0 con 0x7f1a580ce3b0 2024-09-13T16:52:09.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.516+0000 7f1a4f400700 1 -- 172.21.15.3:0/2982657798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1a50014480 con 0x7f1a580ce3b0 2024-09-13T16:52:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a4f400700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a38066730 0x7f1a38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a4f400700 1 -- 172.21.15.3:0/2982657798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1a500849e0 con 0x7f1a580ce3b0 2024-09-13T16:52:09.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a57400700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a38066730 0x7f1a38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a6197d740 1 Processor -- start 2024-09-13T16:52:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a6197d740 1 -- start start 2024-09-13T16:52:09.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a57400700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a38066730 0x7f1a38068bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a48007fd0 tx=0x7f1a48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 0x16d1b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16ce3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1b6c0 0x16ce3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a6197d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c42d80 con 0x16cf9d00 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a6197d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c40510 con 0x16d1ada0 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.520+0000 7f1a6197d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x16c3be00 con 0x16d1b6c0 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16ce3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16ce3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44994/0 (socket says 172.21.15.3:44994) 2024-09-13T16:52:09.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 -- 172.21.15.3:0/306699326 learned_addr learned my addr 172.21.15.3:0/306699326 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a57e00700 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1b6c0 0x16ce3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a57400700 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 0x16d1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 -- 172.21.15.3:0/306699326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1b6c0 msgr2=0x16ce3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1b6c0 0x16ce3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 -- 172.21.15.3:0/306699326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 msgr2=0x16d1b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 0x16d1b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 -- 172.21.15.3:0/306699326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16aae960 con 0x16cf9d00 2024-09-13T16:52:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a56a00700 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16ce3890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009ba0 tx=0x7f1a44010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bd2c08aa37af9b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a43e00700 1 -- 172.21.15.3:0/306699326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44020070 con 0x16cf9d00 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a43e00700 1 -- 172.21.15.3:0/306699326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44011b00 con 0x16cf9d00 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a43e00700 1 -- 172.21.15.3:0/306699326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44011c70 con 0x16cf9d00 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 -- 172.21.15.3:0/306699326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 msgr2=0x16ce3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16ce3890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009ba0 tx=0x7f1a44010000 comp rx=0 tx=0).stop 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 -- 172.21.15.3:0/306699326 shutdown_connections 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1b6c0 0x16ce3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16ce3890 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 --2- 172.21.15.3:0/306699326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 0x16d1b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 -- 172.21.15.3:0/306699326 >> 172.21.15.3:0/306699326 conn(0x16a634d0 msgr2=0x16cfcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 -- 172.21.15.3:0/306699326 shutdown_connections 2024-09-13T16:52:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.524+0000 7f1a6197d740 1 -- 172.21.15.3:0/306699326 wait complete. 2024-09-13T16:52:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 1 -- start start 2024-09-13T16:52:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 1 Processor -- start 2024-09-13T16:52:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16d21290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 0x16d1a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1b6c0 0x16d23ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x16bf9730 con 0x16d1b6c0 2024-09-13T16:52:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x16bff5b0 con 0x16d1ada0 2024-09-13T16:52:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x16ae7aa0 con 0x16cf9d00 2024-09-13T16:52:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a6197d740 10 client.0 _ll_register_callbacks cb 0x16c35610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:52:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1b6c0 0x16d23ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1b6c0 0x16d23ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45006/0 (socket says 172.21.15.3:45006) 2024-09-13T16:52:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 -- 172.21.15.3:0/1510153877 learned_addr learned my addr 172.21.15.3:0/1510153877 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a56a00700 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 0x16d1a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57400700 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16d21290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 -- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 msgr2=0x16d21290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16d21290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 -- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 msgr2=0x16d1a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 0x16d1a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 -- 172.21.15.3:0/1510153877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a580b2080 con 0x16d1b6c0 2024-09-13T16:52:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57400700 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16d21290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a57e00700 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1b6c0 0x16d23ff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c6f0 tx=0x7f1a50055e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0b785385e43110 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a40c00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.528+0000 7f1a40c00700 1 -- 172.21.15.3:0/1510153877 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5006d2d0 con 0x16d1b6c0 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 1 -- 172.21.15.3:0/1510153877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16a701e0 con 0x16d1b6c0 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a40c00700 1 -- 172.21.15.3:0/1510153877 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1a50084040 con 0x16d1b6c0 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a40c00700 1 -- 172.21.15.3:0/1510153877 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a5005dcb0 con 0x16d1b6c0 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 10 client.5547 fetch_fsmap learned FSMap version 16 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 1 -- 172.21.15.3:0/1510153877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x16c02220 con 0x16d1b6c0 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a40c00700 1 -- 172.21.15.3:0/1510153877 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16a701e0 con 0x16d1b6c0 2024-09-13T16:52:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a40c00700 1 -- 172.21.15.3:0/1510153877 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1a5005d820 con 0x16d1b6c0 2024-09-13T16:52:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 10 client.5547 fetch_fsmap finished waiting for FSMap version 16 2024-09-13T16:52:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 10 client.5547 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T16:52:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 20 client.5547 populate_metadata read hostname 'smithi003' 2024-09-13T16:52:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] conn(0x16cd14a0 0x16cd1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 4 client.5547 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:52:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 1 -- 172.21.15.3:0/1510153877 --> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16aba460 con 0x16cd14a0 2024-09-13T16:52:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a57400700 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] conn(0x16cd14a0 0x16cd1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a57400700 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] conn(0x16cd14a0 0x16cd1880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e298c711ae0b3a20 server_cookie=aba62f2aa82f1f0b in_seq=0 out_seq=0 2024-09-13T16:52:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a40c00700 10 client.5547 ms_handle_connect on v2:172.21.15.3:6834/750657669 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a40c00700 1 -- 172.21.15.3:0/1510153877 <== mds.0 v2:172.21.15.3:6834/750657669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x16aba460 con 0x16cd14a0 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a40c00700 10 client.5547 handle_command_reply: tid=0 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.532+0000 7f1a6197d740 1 client.5547 shutdown 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 -- 172.21.15.3:0/1510153877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1b6c0 msgr2=0x16d23ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1b6c0 0x16d23ff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c6f0 tx=0x7f1a50055e00 comp rx=0 tx=0).stop 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 -- 172.21.15.3:0/1510153877 shutdown_connections 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] conn(0x16cd14a0 0x16cd1880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1b6c0 0x16d23ff0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1ada0 0x16d1a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a40c00700 0 client.5547 ms_handle_reset on v2:172.21.15.3:6834/750657669 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1510153877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16d21290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 -- 172.21.15.3:0/1510153877 >> 172.21.15.3:0/1510153877 conn(0x16a634d0 msgr2=0x16cf6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 -- 172.21.15.3:0/1510153877 shutdown_connections 2024-09-13T16:52:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 1 -- 172.21.15.3:0/1510153877 wait complete. 2024-09-13T16:52:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.536+0000 7f1a6197d740 20 client.5547 trim_cache size 0 max 16384 2024-09-13T16:52:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a6197d740 1 Processor -- start 2024-09-13T16:52:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a6197d740 1 -- start start 2024-09-13T16:52:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16cd0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1ada0 0x16d1fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 0x16d20010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a6197d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x16bfde90 con 0x16d1ada0 2024-09-13T16:52:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a6197d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x16be2950 con 0x16d1b6c0 2024-09-13T16:52:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a6197d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x16c3b610 con 0x16cf9d00 2024-09-13T16:52:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a56a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1ada0 0x16d1fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16cd0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16cd0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:47752/0 (socket says 172.21.15.3:47752) 2024-09-13T16:52:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a56a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1ada0 0x16d1fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45018/0 (socket says 172.21.15.3:45018) 2024-09-13T16:52:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 -- 172.21.15.3:0/608710550 learned_addr learned my addr 172.21.15.3:0/608710550 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57e00700 1 --2- 172.21.15.3:0/608710550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 0x16d20010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 -- 172.21.15.3:0/608710550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 msgr2=0x16d20010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 --2- 172.21.15.3:0/608710550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 0x16d20010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 -- 172.21.15.3:0/608710550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1ada0 msgr2=0x16d1fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 --2- 172.21.15.3:0/608710550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1ada0 0x16d1fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 -- 172.21.15.3:0/608710550 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16be8450 con 0x16cf9d00 2024-09-13T16:52:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.556+0000 7f1a57400700 1 --2- 172.21.15.3:0/608710550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16cd0710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a5815bf70 tx=0x7f1a48000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddfc74945aa5249e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a4cc00700 1 -- 172.21.15.3:0/608710550 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480260e0 con 0x16cf9d00 2024-09-13T16:52:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a4cc00700 1 -- 172.21.15.3:0/608710550 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a48026240 con 0x16cf9d00 2024-09-13T16:52:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a4cc00700 1 -- 172.21.15.3:0/608710550 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480263d0 con 0x16cf9d00 2024-09-13T16:52:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 -- 172.21.15.3:0/608710550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 msgr2=0x16cd0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 --2- 172.21.15.3:0/608710550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16cd0710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a5815bf70 tx=0x7f1a48000ed0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 -- 172.21.15.3:0/608710550 shutdown_connections 2024-09-13T16:52:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 --2- 172.21.15.3:0/608710550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 0x16d20010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 --2- 172.21.15.3:0/608710550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16d1ada0 0x16d1fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 --2- 172.21.15.3:0/608710550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16cf9d00 0x16cd0710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 -- 172.21.15.3:0/608710550 >> 172.21.15.3:0/608710550 conn(0x16a634d0 msgr2=0x16cf5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 -- 172.21.15.3:0/608710550 shutdown_connections 2024-09-13T16:52:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 -- 172.21.15.3:0/608710550 wait complete. 2024-09-13T16:52:09.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.560+0000 7f1a6197d740 1 -- start start 2024-09-13T16:52:09.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a6197d740 1 Processor -- start 2024-09-13T16:52:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16cd0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1ada0 0x16cd0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a6197d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 0x16d1f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a6197d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x16cf63f0 con 0x16cf9d00 2024-09-13T16:52:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a6197d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c3c560 con 0x16d1b6c0 2024-09-13T16:52:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a6197d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x16cf6150 con 0x16d1ada0 2024-09-13T16:52:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a6197d740 10 client.0 _ll_register_callbacks cb 0x16c92710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:52:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16cd0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16cd0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45024/0 (socket says 172.21.15.3:45024) 2024-09-13T16:52:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57400700 1 -- 172.21.15.3:0/1503695241 learned_addr learned my addr 172.21.15.3:0/1503695241 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a56a00700 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1ada0 0x16cd0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57e00700 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 0x16d1f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57400700 1 -- 172.21.15.3:0/1503695241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1ada0 msgr2=0x16cd0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57400700 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1ada0 0x16cd0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57400700 1 -- 172.21.15.3:0/1503695241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 msgr2=0x16d1f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57400700 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 0x16d1f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.564+0000 7f1a57400700 1 -- 172.21.15.3:0/1503695241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c0035d0 con 0x16cf9d00 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a57400700 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16cd0910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800f910 tx=0x7f1a480010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15976b21399773fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a40c00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:52:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a40c00700 1 -- 172.21.15.3:0/1503695241 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480267c0 con 0x16cf9d00 2024-09-13T16:52:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a40c00700 1 -- 172.21.15.3:0/1503695241 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1a48004890 con 0x16cf9d00 2024-09-13T16:52:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 1 -- 172.21.15.3:0/1503695241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a44011b00 con 0x16cf9d00 2024-09-13T16:52:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a40c00700 1 -- 172.21.15.3:0/1503695241 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a48026920 con 0x16cf9d00 2024-09-13T16:52:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 10 client.5553 fetch_fsmap learned FSMap version 16 2024-09-13T16:52:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 1 -- 172.21.15.3:0/1503695241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x16bdc720 con 0x16cf9d00 2024-09-13T16:52:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a40c00700 1 -- 172.21.15.3:0/1503695241 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44011b00 con 0x16cf9d00 2024-09-13T16:52:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a40c00700 1 -- 172.21.15.3:0/1503695241 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1a48004a50 con 0x16cf9d00 2024-09-13T16:52:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 10 client.5553 fetch_fsmap finished waiting for FSMap version 16 2024-09-13T16:52:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 10 client.5553 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T16:52:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 20 client.5553 populate_metadata read hostname 'smithi003' 2024-09-13T16:52:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] conn(0x16ccf750 0x16ccfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 4 client.5553 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:52:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a6197d740 1 -- 172.21.15.3:0/1503695241 --> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] -- command(tid 0: {"prefix": "status"}) v1 -- 0x16adf1e0 con 0x16ccf750 2024-09-13T16:52:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.568+0000 7f1a56a00700 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] conn(0x16ccf750 0x16ccfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a56a00700 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] conn(0x16ccf750 0x16ccfb30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c106e4ce5329de1 server_cookie=d381af29802247f0 in_seq=0 out_seq=0 2024-09-13T16:52:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a40c00700 10 client.5553 ms_handle_connect on v2:172.21.15.3:6834/750657669 2024-09-13T16:52:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a40c00700 1 -- 172.21.15.3:0/1503695241 <== mds.0 v2:172.21.15.3:6834/750657669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x16adf1e0 con 0x16ccf750 2024-09-13T16:52:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a40c00700 10 client.5553 handle_command_reply: tid=0 2024-09-13T16:52:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 client.5553 shutdown 2024-09-13T16:52:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 -- 172.21.15.3:0/1503695241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 msgr2=0x16cd0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16cd0910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800f910 tx=0x7f1a480010a0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 -- 172.21.15.3:0/1503695241 shutdown_connections 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.3:6834/750657669,v1:172.21.15.3:6835/750657669] conn(0x16ccf750 0x16ccfb30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x16d1b6c0 0x16d1f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a40c00700 0 client.5553 ms_handle_reset on v2:172.21.15.3:6834/750657669 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x16d1ada0 0x16cd0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 --2- 172.21.15.3:0/1503695241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16cf9d00 0x16cd0910 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 -- 172.21.15.3:0/1503695241 >> 172.21.15.3:0/1503695241 conn(0x16a634d0 msgr2=0x16cf5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 -- 172.21.15.3:0/1503695241 shutdown_connections 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 1 -- 172.21.15.3:0/1503695241 wait complete. 2024-09-13T16:52:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a6197d740 20 client.5553 trim_cache size 0 max 16384 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "61456116-a42d-4065-9bf5-a1daf0be594e", 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5373, 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 6.5212167650000001, 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 16, 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 40, 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 40, 2024-09-13T16:52:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 14.812471224999999 2024-09-13T16:52:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:52:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 -- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a38066730 msgr2=0x7f1a38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a38066730 0x7f1a38068bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a48007fd0 tx=0x7f1a48009040 comp rx=0 tx=0).stop 2024-09-13T16:52:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 -- 172.21.15.3:0/2982657798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a580ce3b0 msgr2=0x7f1a5815a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a580ce3b0 0x7f1a5815a9a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009d20 tx=0x7f1a5000c3a0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 -- 172.21.15.3:0/2982657798 shutdown_connections 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a38066730 0x7f1a38068bf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a580ce3b0 0x7f1a5815a9a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580710b0 0x7f1a581486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 --2- 172.21.15.3:0/2982657798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a5806d0e0 0x7f1a580c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 -- 172.21.15.3:0/2982657798 >> 172.21.15.3:0/2982657798 conn(0x7f1a5806a390 msgr2=0x7f1a580af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 -- 172.21.15.3:0/2982657798 shutdown_connections 2024-09-13T16:52:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.572+0000 7f1a5e600700 1 -- 172.21.15.3:0/2982657798 wait complete. 2024-09-13T16:52:09.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:52:09.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.768+0000 7fbc22a00700 1 Processor -- start 2024-09-13T16:52:09.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.768+0000 7fbc22a00700 1 -- start start 2024-09-13T16:52:09.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.768+0000 7fbc22a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc1c15ce30 0x7fbc1c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.768+0000 7fbc22a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc1c06d720 0x7fbc1c0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc22a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c06ce30 con 0x7fbc1c06ba80 2024-09-13T16:52:09.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc22a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c15d210 con 0x7fbc1c15ce30 2024-09-13T16:52:09.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc22a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c0b1ef0 con 0x7fbc1c06d720 2024-09-13T16:52:09.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc21600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc1c15ce30 0x7fbc1c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc22000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbc1c06ba80 0x7fbc1c06a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45282/0 (socket says 172.21.15.3:45282) 2024-09-13T16:52:09.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc22000700 1 -- 172.21.15.3:0/775180912 learned_addr learned my addr 172.21.15.3:0/775180912 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:09.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc20c00700 1 --2- 172.21.15.3:0/775180912 >> v2:172.21.15.157:3300/0 conn(0x7fbc1c06d720 0x7fbc1c0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc20c00700 1 -- 172.21.15.3:0/775180912 >> v2:172.21.15.3:3300/0 conn(0x7fbc1c15ce30 msgr2=0x7fbc1c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc20c00700 1 --2- 172.21.15.3:0/775180912 >> v2:172.21.15.3:3300/0 conn(0x7fbc1c15ce30 0x7fbc1c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc20c00700 1 -- 172.21.15.3:0/775180912 >> v1:172.21.15.3:6789/0 conn(0x7fbc1c06ba80 legacy=0x7fbc1c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc20c00700 1 -- 172.21.15.3:0/775180912 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c06d240 con 0x7fbc1c06d720 2024-09-13T16:52:09.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc1be00700 1 -- 172.21.15.3:0/775180912 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211485350 0 0) 0x7fbc1c06ce30 con 0x7fbc1c06ba80 2024-09-13T16:52:09.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc20c00700 1 --2- 172.21.15.3:0/775180912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c06d720 0x7fbc1c0b1510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c070eb0 tx=0x7fbc08009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0249539a31a93d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc1be00700 1 -- 172.21.15.3:0/775180912 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08011830 con 0x7fbc1c06d720 2024-09-13T16:52:09.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc1be00700 1 -- 172.21.15.3:0/775180912 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc08011990 con 0x7fbc1c06d720 2024-09-13T16:52:09.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc1be00700 1 -- 172.21.15.3:0/775180912 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08011d20 con 0x7fbc1c06d720 2024-09-13T16:52:09.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc22a00700 1 -- 172.21.15.3:0/775180912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c06d720 msgr2=0x7fbc1c0b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.772+0000 7fbc22a00700 1 --2- 172.21.15.3:0/775180912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c06d720 0x7fbc1c0b1510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c070eb0 tx=0x7fbc08009b30 comp rx=0 tx=0).stop 2024-09-13T16:52:09.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.776+0000 7fbc22a00700 1 -- 172.21.15.3:0/775180912 shutdown_connections 2024-09-13T16:52:09.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.776+0000 7fbc22a00700 1 --2- 172.21.15.3:0/775180912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c06d720 0x7fbc1c0b1510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.776+0000 7fbc22a00700 1 --2- 172.21.15.3:0/775180912 >> v2:172.21.15.3:3300/0 conn(0x7fbc1c15ce30 0x7fbc1c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.776+0000 7fbc22a00700 1 -- 172.21.15.3:0/775180912 >> 172.21.15.3:0/775180912 conn(0x7fbc1c0b4d70 msgr2=0x7fbc1c0b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.776+0000 7fbc22a00700 1 -- 172.21.15.3:0/775180912 shutdown_connections 2024-09-13T16:52:09.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.776+0000 7fbc22a00700 1 -- 172.21.15.3:0/775180912 wait complete. 2024-09-13T16:52:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.776+0000 7fbc22a00700 1 Processor -- start 2024-09-13T16:52:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.776+0000 7fbc22a00700 1 -- start start 2024-09-13T16:52:09.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc1c06ba80 0x7fbc1c157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc1c06d720 0x7fbc1c150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c15ce30 0x7fbc1c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c09b830 con 0x7fbc1c06d720 2024-09-13T16:52:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c1505b0 con 0x7fbc1c15ce30 2024-09-13T16:52:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c09a9c0 con 0x7fbc1c06ba80 2024-09-13T16:52:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc1c06d720 0x7fbc1c150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc1c06d720 0x7fbc1c150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45044/0 (socket says 172.21.15.3:45044) 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 -- 172.21.15.3:0/1402765282 learned_addr learned my addr 172.21.15.3:0/1402765282 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc21600700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc1c06ba80 0x7fbc1c157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22000700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c15ce30 0x7fbc1c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 -- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc1c06ba80 msgr2=0x7fbc1c157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc1c06ba80 0x7fbc1c157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 -- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c15ce30 msgr2=0x7fbc1c09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c15ce30 0x7fbc1c09a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 -- 172.21.15.3:0/1402765282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc080094b0 con 0x7fbc1c06d720 2024-09-13T16:52:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc21600700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc1c06ba80 0x7fbc1c157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc20c00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc1c06d720 0x7fbc1c150070 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000c00 tx=0x7fbc08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc19600700 1 -- 172.21.15.3:0/1402765282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08004450 con 0x7fbc1c06d720 2024-09-13T16:52:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc19600700 1 -- 172.21.15.3:0/1402765282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0801f070 con 0x7fbc1c06d720 2024-09-13T16:52:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc19600700 1 -- 172.21.15.3:0/1402765282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08003900 con 0x7fbc1c06d720 2024-09-13T16:52:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c09ac20 con 0x7fbc1c06d720 2024-09-13T16:52:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.780+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c09b1e0 con 0x7fbc1c06d720 2024-09-13T16:52:09.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.784+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c09b9a0 con 0x7fbc1c06d720 2024-09-13T16:52:09.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.784+0000 7fbc19600700 1 -- 172.21.15.3:0/1402765282 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc08003a60 con 0x7fbc1c06d720 2024-09-13T16:52:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.788+0000 7fbc19600700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbbfc066760 0x7fbbfc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.788+0000 7fbc19600700 1 -- 172.21.15.3:0/1402765282 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbc08021070 con 0x7fbc1c06d720 2024-09-13T16:52:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.788+0000 7fbc21600700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbbfc066760 0x7fbbfc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.788+0000 7fbc21600700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbbfc066760 0x7fbbfc068c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009f40 tx=0x7fbc10009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:09.796+0000 7fbc19600700 1 -- 172.21.15.3:0/1402765282 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbc08052b10 con 0x7fbc1c06d720 2024-09-13T16:52:10.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.028+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc1c0b0fd0 con 0x7fbc1c06d720 2024-09-13T16:52:10.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.032+0000 7fbc19600700 1 -- 172.21.15.3:0/1402765282 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbc0802b070 con 0x7fbc1c06d720 2024-09-13T16:52:10.036 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T16:52:10.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbbfc066760 msgr2=0x7fbbfc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:10.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbbfc066760 0x7fbbfc068c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009f40 tx=0x7fbc10009240 comp rx=0 tx=0).stop 2024-09-13T16:52:10.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc1c06d720 msgr2=0x7fbc1c150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:10.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc1c06d720 0x7fbc1c150070 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000c00 tx=0x7fbc08004770 comp rx=0 tx=0).stop 2024-09-13T16:52:10.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 shutdown_connections 2024-09-13T16:52:10.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbbfc066760 0x7fbbfc068c20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:10.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc1c15ce30 0x7fbc1c09a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:10.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc1c06d720 0x7fbc1c150070 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 --2- 172.21.15.3:0/1402765282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc1c06ba80 0x7fbc1c157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 >> 172.21.15.3:0/1402765282 conn(0x7fbc1c0b4d70 msgr2=0x7fbc1c15f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 shutdown_connections 2024-09-13T16:52:10.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:10.036+0000 7fbc22a00700 1 -- 172.21.15.3:0/1402765282 wait complete. 2024-09-13T16:52:10.065 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T16:52:10.065 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:52:10.065 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-13T16:52:10.065 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T16:52:10.065 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T16:52:10.065 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T16:52:10.065 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T16:52:10.066 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:52:10.066 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:52:10.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.066 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-13T16:52:10.083 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:52:10.083 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:52:10.083 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:52:10.083 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-13T16:52:10.084 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42335sec preferred_lft 42335sec 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:4d:e8:d2:67 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether f2:5f:fd:43:95:5c brd ff:ff:ff:ff:ff:ff 2024-09-13T16:52:10.085 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::4cad:7aff:fedc:96c0/64 scope link 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether f2:5f:fd:43:95:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f05f:fdff:fe43:955c/64 scope link 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether fa:f0:e1:81:09:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f8f0:e1ff:fe81:969/64 scope link 2024-09-13T16:52:10.086 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:52:10.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.087 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:52:10.167 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-13T16:52:10.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.170 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-13T16:52:10.238 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-13T16:52:10.239 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:52:10.239 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T16:52:10.239 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:52:10.239 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T16:52:10.239 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:52:10.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.240 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:52:10.240 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:52:10.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T16:52:10.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T16:52:10.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T16:52:10.240 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:52:10.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.380 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:52:10.395 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T16:52:10.395 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T16:52:10.396 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:52:10.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-13T16:52:10.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:52:10.480 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T16:52:10.546 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:52:10.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:52:10.547 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:52:10.608 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-13T16:52:10.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:52:10.741 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:52:10.741 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:52:10.768 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:52:10.768 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:52:10.768 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-13T16:52:10.768 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 0@61456116-a42d-4065-9bf5-a1daf0be594e.cephfs=/ 2024-09-13T16:52:10.768 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.3:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-13T16:52:10.775 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T16:52:10.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:52:10.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:10.806 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:52:10.881 INFO:teuthology.orchestra.run.smithi003.stdout:client5141 2024-09-13T16:52:10.881 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:10.881 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:10.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:52:10.989 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-13T16:52:11.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.061 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:52:11.062 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.063 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-13T16:52:11.064 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.068 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-13T16:52:11.068 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-13T16:52:11.068 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:52:11.068 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.068 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.069 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2024-09-13T16:52:11.070 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:52:11.071 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-13T16:52:11.071 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:52:11.071 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:52:11.071 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-13T16:52:11.071 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:52:11.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.071 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-13T16:52:11.071 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-13T16:52:11.072 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:52:11.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-13T16:52:11.074 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-13T16:52:11.075 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:52:11.076 INFO:teuthology.orchestra.run.smithi003.stdout: osd_deep_scrub_large_omap_object_key_threshold = 5 2024-09-13T16:52:11.077 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-09-13T16:52:11.077 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-09-13T16:52:11.077 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:52:11.077 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-09-13T16:52:11.077 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.077 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-09-13T16:52:11.077 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:52:11.078 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.079 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.080 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-09-13T16:52:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:52:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:52:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-09-13T16:52:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:52:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.084 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-09-13T16:52:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-09-13T16:52:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-09-13T16:52:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:52:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on pool no app = False 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:52:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-09-13T16:52:11.088 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:52:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: osd_deep_scrub_large_omap_object_key_threshold = 5 2024-09-13T16:52:11.091 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:52:11.091 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T16:52:11.091 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:52:11.091 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:11.092 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T16:52:11.092 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T16:52:11.092 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T16:52:11.092 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:11.092 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:52:11.092 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T16:52:11.092 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:52:11.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:11.093 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:52:11.093 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T16:52:11.093 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:52:11.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:11.110 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:52:11.110 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:52:11.113 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:52:11.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:52:11.117 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:52:11.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:52:11.119 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T16:52:11.119 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T16:52:11.164 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:52:11.165 DEBUG:teuthology.parallel:result is None 2024-09-13T16:52:11.168 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:52:11.168 DEBUG:teuthology.parallel:result is None 2024-09-13T16:52:11.169 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:52:11.170 DEBUG:teuthology.parallel:result is None 2024-09-13T16:52:11.172 INFO:tasks.ceph.mds.c:Started 2024-09-13T16:52:11.173 DEBUG:teuthology.parallel:result is None 2024-09-13T16:52:11.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:11.285 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-13T16:52:11.292 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-13T16:52:11.293 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-13T16:52:11.294 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-13T16:52:11.312 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-13T16:52:11.308+0000 7f9c47a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:52:11.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6c00700 1 Processor -- start 2024-09-13T16:52:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6c00700 1 -- start start 2024-09-13T16:52:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29c015ed00 0x7f29c01621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29c00c3980 0x7f29c00c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c01651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00b01d0 con 0x7f29c00c3980 2024-09-13T16:52:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00c4350 con 0x7f29c015ed00 2024-09-13T16:52:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0162700 con 0x7f29c0164d30 2024-09-13T16:52:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c4e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29c00c3980 0x7f29c00c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c5800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29c015ed00 0x7f29c01621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c6200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c01651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c4e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29c00c3980 0x7f29c00c3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45072/0 (socket says 172.21.15.3:45072) 2024-09-13T16:52:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.336+0000 7f29c4e00700 1 -- 172.21.15.3:0/1470380830 learned_addr learned my addr 172.21.15.3:0/1470380830 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c4e00700 1 -- 172.21.15.3:0/1470380830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 msgr2=0x7f29c01651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c4e00700 1 --2- 172.21.15.3:0/1470380830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c01651c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c4e00700 1 -- 172.21.15.3:0/1470380830 >> v2:172.21.15.157:3300/0 conn(0x7f29c015ed00 msgr2=0x7f29c01621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c4e00700 1 --2- 172.21.15.3:0/1470380830 >> v2:172.21.15.157:3300/0 conn(0x7f29c015ed00 0x7f29c01621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c4e00700 1 -- 172.21.15.3:0/1470380830 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00afef0 con 0x7f29c00c3980 2024-09-13T16:52:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c4e00700 1 --2- 172.21.15.3:0/1470380830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 0x7f29c00c3e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f29c0071040 tx=0x7f29ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a807c1828a4670c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29bfe00700 1 -- 172.21.15.3:0/1470380830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac011830 con 0x7f29c00c3980 2024-09-13T16:52:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29bfe00700 1 -- 172.21.15.3:0/1470380830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac011990 con 0x7f29c00c3980 2024-09-13T16:52:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29bfe00700 1 -- 172.21.15.3:0/1470380830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac011d20 con 0x7f29c00c3980 2024-09-13T16:52:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 -- 172.21.15.3:0/1470380830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 msgr2=0x7f29c00c3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 --2- 172.21.15.3:0/1470380830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 0x7f29c00c3e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f29c0071040 tx=0x7f29ac009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 -- 172.21.15.3:0/1470380830 shutdown_connections 2024-09-13T16:52:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 --2- 172.21.15.3:0/1470380830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 0x7f29c00c3e10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 --2- 172.21.15.3:0/1470380830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c01651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 --2- 172.21.15.3:0/1470380830 >> v2:172.21.15.157:3300/0 conn(0x7f29c015ed00 0x7f29c01621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 -- 172.21.15.3:0/1470380830 >> 172.21.15.3:0/1470380830 conn(0x7f29c00b24c0 msgr2=0x7f29c00b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 -- 172.21.15.3:0/1470380830 shutdown_connections 2024-09-13T16:52:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.340+0000 7f29c6c00700 1 -- 172.21.15.3:0/1470380830 wait complete. 2024-09-13T16:52:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.344+0000 7f29c6c00700 1 Processor -- start 2024-09-13T16:52:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.344+0000 7f29c6c00700 1 -- start start 2024-09-13T16:52:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.344+0000 7f29c6c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 0x7f29c01603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.344+0000 7f29c6c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015ed00 0x7f29c00aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.344+0000 7f29c6c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c00af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.344+0000 7f29c6c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c014d9d0 con 0x7f29c00c3980 2024-09-13T16:52:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.344+0000 7f29c6c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c7340 con 0x7f29c015ed00 2024-09-13T16:52:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.344+0000 7f29c6c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c66c0 con 0x7f29c0164d30 2024-09-13T16:52:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c00af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c00af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:47788/0 (socket says 172.21.15.3:47788) 2024-09-13T16:52:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 -- 172.21.15.3:0/479051783 learned_addr learned my addr 172.21.15.3:0/479051783 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c5800700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 0x7f29c01603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:11.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c4e00700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015ed00 0x7f29c00aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 -- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015ed00 msgr2=0x7f29c00aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015ed00 0x7f29c00aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 -- 172.21.15.3:0/479051783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 msgr2=0x7f29c01603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 0x7f29c01603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 -- 172.21.15.3:0/479051783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac009570 con 0x7f29c0164d30 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c5800700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 0x7f29c01603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6200700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c00af2d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29b800efc0 tx=0x7f29b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29bd600700 1 -- 172.21.15.3:0/479051783 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8009ea0 con 0x7f29c0164d30 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29bd600700 1 -- 172.21.15.3:0/479051783 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b800f040 con 0x7f29c0164d30 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29bd600700 1 -- 172.21.15.3:0/479051783 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8014480 con 0x7f29c0164d30 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c00c69a0 con 0x7f29c0164d30 2024-09-13T16:52:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.348+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c00c6f60 con 0x7f29c0164d30 2024-09-13T16:52:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.352+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c00c7480 con 0x7f29c0164d30 2024-09-13T16:52:11.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.352+0000 7f29bd600700 1 -- 172.21.15.3:0/479051783 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29b80145e0 con 0x7f29c0164d30 2024-09-13T16:52:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.352+0000 7f29bd600700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f29a0066730 0x7f29a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.352+0000 7f29bd600700 1 -- 172.21.15.3:0/479051783 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f29b8084b10 con 0x7f29c0164d30 2024-09-13T16:52:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.352+0000 7f29c5800700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f29a0066730 0x7f29a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:11.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.352+0000 7f29c5800700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f29a0066730 0x7f29a0068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009bc0 tx=0x7f29b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.364+0000 7f29bd600700 1 -- 172.21.15.3:0/479051783 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f29c00c7480 con 0x7f29c0164d30 2024-09-13T16:52:11.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.588+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29c00c7480 con 0x7f29c0164d30 2024-09-13T16:52:11.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.588+0000 7f29bd600700 1 -- 172.21.15.3:0/479051783 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f29c00c7480 con 0x7f29c0164d30 2024-09-13T16:52:11.595 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:11.595 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:11.324539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5569":{"gid":5569,"name":"c","rank":0,"incarnation":18,"state":"up:replay","state_seq":1,"addr":"172.21.15.3:6835/1311223400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1311223400},{"type":"v1","addr":"172.21.15.3:6835","nonce":1311223400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.592+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f29a0066730 msgr2=0x7f29a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.592+0000 7f29c6c00700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f29a0066730 0x7f29a0068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009bc0 tx=0x7f29b4008040 comp rx=0 tx=0).stop 2024-09-13T16:52:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.592+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 msgr2=0x7f29c00af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.592+0000 7f29c6c00700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c00af2d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29b800efc0 tx=0x7f29b800c3a0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.596+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 shutdown_connections 2024-09-13T16:52:11.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.596+0000 7f29c6c00700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f29a0066730 0x7f29a0068bf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.596+0000 7f29c6c00700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c0164d30 0x7f29c00af2d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.596+0000 7f29c6c00700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015ed00 0x7f29c00aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.596+0000 7f29c6c00700 1 --2- 172.21.15.3:0/479051783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c00c3980 0x7f29c01603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:11.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.596+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 >> 172.21.15.3:0/479051783 conn(0x7f29c00b24c0 msgr2=0x7f29c00ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:11.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.596+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 shutdown_connections 2024-09-13T16:52:11.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:11.596+0000 7f29c6c00700 1 -- 172.21.15.3:0/479051783 wait complete. 2024-09-13T16:52:11.601 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 18 2024-09-13T16:52:12.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:52:03.049167+0000', 'modified': '2024-09-13T16:52:11.324539+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5569': {'gid': 5569, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.3:6835/1311223400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6832', 'nonce': 1311223400}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1311223400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:52:12.024 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5569:up:replay 2024-09-13T16:52:13.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:13.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.196+0000 7fea72e00700 1 Processor -- start 2024-09-13T16:52:13.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.196+0000 7fea72e00700 1 -- start start 2024-09-13T16:52:13.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea6c1652e0 0x7fea6c16bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:13.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6c06ba80 con 0x7fea6c160f30 2024-09-13T16:52:13.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6c06fad0 con 0x7fea6c165770 2024-09-13T16:52:13.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6c06bcd0 con 0x7fea6c1652e0 2024-09-13T16:52:13.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea71a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea6c160f30 0x7fea6c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45284/0 (socket says 172.21.15.3:45284) 2024-09-13T16:52:13.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea71a00700 1 -- 172.21.15.3:0/3092697009 learned_addr learned my addr 172.21.15.3:0/3092697009 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72400700 1 --2- 172.21.15.3:0/3092697009 >> v2:172.21.15.157:3300/0 conn(0x7fea6c1652e0 0x7fea6c16bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea6be00700 1 -- 172.21.15.3:0/3092697009 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630414429 0 0) 0x7fea6c06ba80 con 0x7fea6c160f30 2024-09-13T16:52:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea6be00700 1 -- 172.21.15.3:0/3092697009 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea540035d0 con 0x7fea6c160f30 2024-09-13T16:52:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea6be00700 1 -- 172.21.15.3:0/3092697009 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 935490021 0 0) 0x7fea540035d0 con 0x7fea6c160f30 2024-09-13T16:52:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72400700 1 -- 172.21.15.3:0/3092697009 >> v1:172.21.15.157:6789/0 conn(0x7fea6c165770 legacy=0x7fea6c164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72400700 1 -- 172.21.15.3:0/3092697009 >> v1:172.21.15.3:6789/0 conn(0x7fea6c160f30 legacy=0x7fea6c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72400700 1 -- 172.21.15.3:0/3092697009 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c06bfb0 con 0x7fea6c1652e0 2024-09-13T16:52:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea72400700 1 --2- 172.21.15.3:0/3092697009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c1652e0 0x7fea6c16bd50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea64009d90 tx=0x7fea6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f76d9f569e967 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea6be00700 1 -- 172.21.15.3:0/3092697009 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6400e070 con 0x7fea6c1652e0 2024-09-13T16:52:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea6be00700 1 -- 172.21.15.3:0/3092697009 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64003a40 con 0x7fea6c1652e0 2024-09-13T16:52:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.200+0000 7fea6be00700 1 -- 172.21.15.3:0/3092697009 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60001670 con 0x7fea6c1652e0 2024-09-13T16:52:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 -- 172.21.15.3:0/3092697009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c1652e0 msgr2=0x7fea6c16bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 --2- 172.21.15.3:0/3092697009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c1652e0 0x7fea6c16bd50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea64009d90 tx=0x7fea6400b330 comp rx=0 tx=0).stop 2024-09-13T16:52:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 -- 172.21.15.3:0/3092697009 shutdown_connections 2024-09-13T16:52:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 --2- 172.21.15.3:0/3092697009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c1652e0 0x7fea6c16bd50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 -- 172.21.15.3:0/3092697009 >> 172.21.15.3:0/3092697009 conn(0x7fea6c06a2f0 msgr2=0x7fea6c06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 -- 172.21.15.3:0/3092697009 shutdown_connections 2024-09-13T16:52:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 -- 172.21.15.3:0/3092697009 wait complete. 2024-09-13T16:52:13.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 Processor -- start 2024-09-13T16:52:13.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.204+0000 7fea72e00700 1 -- start start 2024-09-13T16:52:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c160f30 0x7fea6c1490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6c1652e0 0x7fea6c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea6c165770 0x7fea6c15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c162030 con 0x7fea6c1652e0 2024-09-13T16:52:13.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c154f20 con 0x7fea6c160f30 2024-09-13T16:52:13.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1622d0 con 0x7fea6c165770 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6c1652e0 0x7fea6c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6c1652e0 0x7fea6c1544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45102/0 (socket says 172.21.15.3:45102) 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c160f30 0x7fea6c1490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 -- 172.21.15.3:0/2232324149 learned_addr learned my addr 172.21.15.3:0/2232324149 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72400700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea6c165770 0x7fea6c15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 -- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea6c165770 msgr2=0x7fea6c15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea6c165770 0x7fea6c15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 -- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c160f30 msgr2=0x7fea6c1490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c160f30 0x7fea6c1490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 -- 172.21.15.3:0/2232324149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64009a70 con 0x7fea6c1652e0 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71a00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c160f30 0x7fea6c1490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea71000700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6c1652e0 0x7fea6c1544a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c500 tx=0x7fea5800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:13.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea69600700 1 -- 172.21.15.3:0/2232324149 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580052d0 con 0x7fea6c1652e0 2024-09-13T16:52:13.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea69600700 1 -- 172.21.15.3:0/2232324149 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea58005430 con 0x7fea6c1652e0 2024-09-13T16:52:13.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea69600700 1 -- 172.21.15.3:0/2232324149 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580055a0 con 0x7fea6c1652e0 2024-09-13T16:52:13.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c1625b0 con 0x7fea6c1652e0 2024-09-13T16:52:13.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.208+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c149b60 con 0x7fea6c1652e0 2024-09-13T16:52:13.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.212+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c1627f0 con 0x7fea6c1652e0 2024-09-13T16:52:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.212+0000 7fea69600700 1 -- 172.21.15.3:0/2232324149 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea580038e0 con 0x7fea6c1652e0 2024-09-13T16:52:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.216+0000 7fea69600700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea4c066730 0x7fea4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.216+0000 7fea69600700 1 -- 172.21.15.3:0/2232324149 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fea58083650 con 0x7fea6c1652e0 2024-09-13T16:52:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.216+0000 7fea71a00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea4c066730 0x7fea4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:13.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.216+0000 7fea71a00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea4c066730 0x7fea4c068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea60001eb0 tx=0x7fea60001e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.224+0000 7fea69600700 1 -- 172.21.15.3:0/2232324149 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fea6c1627f0 con 0x7fea6c1652e0 2024-09-13T16:52:13.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.456+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea6c1627a0 con 0x7fea6c1652e0 2024-09-13T16:52:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea69600700 1 -- 172.21.15.3:0/2232324149 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fea58050810 con 0x7fea6c1652e0 2024-09-13T16:52:13.464 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:13.464 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:13.340786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5569":{"gid":5569,"name":"c","rank":0,"incarnation":18,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.3:6835/1311223400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1311223400},{"type":"v1","addr":"172.21.15.3:6835","nonce":1311223400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea4c066730 msgr2=0x7fea4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea4c066730 0x7fea4c068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea60001eb0 tx=0x7fea60001e40 comp rx=0 tx=0).stop 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6c1652e0 msgr2=0x7fea6c1544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6c1652e0 0x7fea6c1544a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c500 tx=0x7fea5800c810 comp rx=0 tx=0).stop 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 shutdown_connections 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fea4c066730 0x7fea4c068bf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea6c165770 0x7fea6c15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6c1652e0 0x7fea6c1544a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:13.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 --2- 172.21.15.3:0/2232324149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c160f30 0x7fea6c1490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:13.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 >> 172.21.15.3:0/2232324149 conn(0x7fea6c06a2f0 msgr2=0x7fea6c0ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:13.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 shutdown_connections 2024-09-13T16:52:13.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:13.460+0000 7fea72e00700 1 -- 172.21.15.3:0/2232324149 wait complete. 2024-09-13T16:52:13.469 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 20 2024-09-13T16:52:13.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:52:03.049167+0000', 'modified': '2024-09-13T16:52:13.340786+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5569': {'gid': 5569, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.3:6835/1311223400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6832', 'nonce': 1311223400}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1311223400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:52:13.892 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5569:up:rejoin 2024-09-13T16:52:14.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:15.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f1361800700 1 Processor -- start 2024-09-13T16:52:15.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f1361800700 1 -- start start 2024-09-13T16:52:15.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f1361800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f135c06ba80 0x7f135c06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:15.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f1361800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f135c06be60 0x7f135c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:15.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f1361800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135c0ce5d0 con 0x7f135c06fdb0 2024-09-13T16:52:15.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f1361800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135c15af00 con 0x7f135c06ba80 2024-09-13T16:52:15.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f1361800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135c0cd9c0 con 0x7f135c06be60 2024-09-13T16:52:15.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f135be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f135c06ba80 0x7f135c06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:15.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f135be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f135c06ba80 0x7f135c06d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56010/0 (socket says 172.21.15.3:56010) 2024-09-13T16:52:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f135be00700 1 -- 172.21.15.3:0/56345766 learned_addr learned my addr 172.21.15.3:0/56345766 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f135b400700 1 --2- 172.21.15.3:0/56345766 >> v2:172.21.15.157:3300/0 conn(0x7f135c06be60 0x7f135c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f135be00700 1 -- 172.21.15.3:0/56345766 >> v2:172.21.15.157:3300/0 conn(0x7f135c06be60 msgr2=0x7f135c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f135be00700 1 --2- 172.21.15.3:0/56345766 >> v2:172.21.15.157:3300/0 conn(0x7f135c06be60 0x7f135c06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f135be00700 1 -- 172.21.15.3:0/56345766 >> v1:172.21.15.3:6789/0 conn(0x7f135c06fdb0 legacy=0x7f135c0701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.064+0000 7f135be00700 1 -- 172.21.15.3:0/56345766 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c0cdca0 con 0x7f135c06ba80 2024-09-13T16:52:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f135be00700 1 --2- 172.21.15.3:0/56345766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c06d6b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ae2d0 tx=0x7f134c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bf2731f25144e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f135aa00700 1 -- 172.21.15.3:0/56345766 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c00b8a0 con 0x7f135c06ba80 2024-09-13T16:52:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f135aa00700 1 -- 172.21.15.3:0/56345766 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c00ba00 con 0x7f135c06ba80 2024-09-13T16:52:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f135aa00700 1 -- 172.21.15.3:0/56345766 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c00bd90 con 0x7f135c06ba80 2024-09-13T16:52:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f1361800700 1 -- 172.21.15.3:0/56345766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 msgr2=0x7f135c06d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:15.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f1361800700 1 --2- 172.21.15.3:0/56345766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c06d6b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ae2d0 tx=0x7f134c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f1361800700 1 -- 172.21.15.3:0/56345766 shutdown_connections 2024-09-13T16:52:15.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f1361800700 1 --2- 172.21.15.3:0/56345766 >> v2:172.21.15.157:3300/0 conn(0x7f135c06be60 0x7f135c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f1361800700 1 --2- 172.21.15.3:0/56345766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c06d6b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f1361800700 1 -- 172.21.15.3:0/56345766 >> 172.21.15.3:0/56345766 conn(0x7f135c06a2f0 msgr2=0x7f135c06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:15.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f1361800700 1 -- 172.21.15.3:0/56345766 shutdown_connections 2024-09-13T16:52:15.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.068+0000 7f1361800700 1 -- 172.21.15.3:0/56345766 wait complete. 2024-09-13T16:52:15.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1361800700 1 Processor -- start 2024-09-13T16:52:15.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1361800700 1 -- start start 2024-09-13T16:52:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1361800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1361800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f135c06be60 0x7f135c153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1361800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135c06fdb0 0x7f135c147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1361800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c14bdb0 con 0x7f135c06ba80 2024-09-13T16:52:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1361800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c0b3770 con 0x7f135c06fdb0 2024-09-13T16:52:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1361800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135c148610 con 0x7f135c06be60 2024-09-13T16:52:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56016/0 (socket says 172.21.15.3:56016) 2024-09-13T16:52:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 -- 172.21.15.3:0/4068900283 learned_addr learned my addr 172.21.15.3:0/4068900283 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 -- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f135c06be60 msgr2=0x7f135c153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135b400700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f135c06be60 0x7f135c153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1360e00700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135c06fdb0 0x7f135c147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f135c06be60 0x7f135c153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 -- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135c06fdb0 msgr2=0x7f135c147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135c06fdb0 0x7f135c147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 -- 172.21.15.3:0/4068900283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c009570 con 0x7f135c06ba80 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135b400700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f135c06be60 0x7f135c153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f1360e00700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135c06fdb0 0x7f135c147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.072+0000 7f135be00700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c0b3230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f135c072390 tx=0x7f134c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.076+0000 7f1353e00700 1 -- 172.21.15.3:0/4068900283 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0038c0 con 0x7f135c06ba80 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.076+0000 7f1353e00700 1 -- 172.21.15.3:0/4068900283 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c01f070 con 0x7f135c06ba80 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.076+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135c1488f0 con 0x7f135c06ba80 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.076+0000 7f1353e00700 1 -- 172.21.15.3:0/4068900283 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c003c60 con 0x7f135c06ba80 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.076+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c152a60 con 0x7f135c06ba80 2024-09-13T16:52:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.076+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135c06c2a0 con 0x7f135c06ba80 2024-09-13T16:52:15.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.076+0000 7f1353e00700 1 -- 172.21.15.3:0/4068900283 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f134c026480 con 0x7f135c06ba80 2024-09-13T16:52:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.080+0000 7f1353e00700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f133c066730 0x7f133c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.080+0000 7f1353e00700 1 -- 172.21.15.3:0/4068900283 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f134c021070 con 0x7f135c06ba80 2024-09-13T16:52:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.080+0000 7f135b400700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f133c066730 0x7f133c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:15.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.080+0000 7f135b400700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f133c066730 0x7f133c068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f135c1582f0 tx=0x7f1348008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.084+0000 7f1353e00700 1 -- 172.21.15.3:0/4068900283 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f134c053080 con 0x7f135c06ba80 2024-09-13T16:52:15.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.312+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f135c0cbe40 con 0x7f135c06ba80 2024-09-13T16:52:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.316+0000 7f1353e00700 1 -- 172.21.15.3:0/4068900283 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f134c015090 con 0x7f135c06ba80 2024-09-13T16:52:15.321 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:15.321 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:14.351471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5569":{"gid":5569,"name":"c","rank":0,"incarnation":18,"state":"up:clientreplay","state_seq":4,"addr":"172.21.15.3:6835/1311223400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1311223400},{"type":"v1","addr":"172.21.15.3:6835","nonce":1311223400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:15.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f133c066730 msgr2=0x7f133c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:15.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f133c066730 0x7f133c068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f135c1582f0 tx=0x7f1348008040 comp rx=0 tx=0).stop 2024-09-13T16:52:15.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 msgr2=0x7f135c0b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:15.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c0b3230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f135c072390 tx=0x7f134c004770 comp rx=0 tx=0).stop 2024-09-13T16:52:15.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 shutdown_connections 2024-09-13T16:52:15.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f133c066730 0x7f133c068bf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135c06fdb0 0x7f135c147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f135c06be60 0x7f135c153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 --2- 172.21.15.3:0/4068900283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f135c06ba80 0x7f135c0b3230 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:15.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 >> 172.21.15.3:0/4068900283 conn(0x7f135c06a2f0 msgr2=0x7f135c06cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:15.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 shutdown_connections 2024-09-13T16:52:15.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:15.320+0000 7f1361800700 1 -- 172.21.15.3:0/4068900283 wait complete. 2024-09-13T16:52:15.330 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-09-13T16:52:15.757 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:52:03.049167+0000', 'modified': '2024-09-13T16:52:14.351471+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5569': {'gid': 5569, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:clientreplay', 'state_seq': 4, 'addr': '172.21.15.3:6835/1311223400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6832', 'nonce': 1311223400}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1311223400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:52:15.757 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5569:up:clientreplay 2024-09-13T16:52:16.347 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-13T16:52:16.340+0000 7fda20c00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:52:16.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:16.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce5d600700 1 Processor -- start 2024-09-13T16:52:16.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce5d600700 1 -- start start 2024-09-13T16:52:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce5d600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce5815d5b0 0x7fce5806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce5d600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce580cc8f0 0x7fce5815d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce5d600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5806de50 con 0x7fce58159fc0 2024-09-13T16:52:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce5d600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580ccd80 con 0x7fce5815d5b0 2024-09-13T16:52:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce5d600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5815ded0 con 0x7fce580cc8f0 2024-09-13T16:52:16.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce56a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce5815d5b0 0x7fce5806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce56a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce5815d5b0 0x7fce5806d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56020/0 (socket says 172.21.15.3:56020) 2024-09-13T16:52:16.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce56a00700 1 -- 172.21.15.3:0/2004580644 learned_addr learned my addr 172.21.15.3:0/2004580644 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:16.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.932+0000 7fce56000700 1 --2- 172.21.15.3:0/2004580644 >> v2:172.21.15.157:3300/0 conn(0x7fce580cc8f0 0x7fce5815d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce56a00700 1 -- 172.21.15.3:0/2004580644 >> v2:172.21.15.157:3300/0 conn(0x7fce580cc8f0 msgr2=0x7fce5815d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce56a00700 1 --2- 172.21.15.3:0/2004580644 >> v2:172.21.15.157:3300/0 conn(0x7fce580cc8f0 0x7fce5815d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce56a00700 1 -- 172.21.15.3:0/2004580644 >> v1:172.21.15.3:6789/0 conn(0x7fce58159fc0 legacy=0x7fce5815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce56a00700 1 -- 172.21.15.3:0/2004580644 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5806db70 con 0x7fce5815d5b0 2024-09-13T16:52:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce56000700 1 --2- 172.21.15.3:0/2004580644 >> v2:172.21.15.157:3300/0 conn(0x7fce580cc8f0 0x7fce5815d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce56a00700 1 --2- 172.21.15.3:0/2004580644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce5815d5b0 0x7fce5806d4b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fce58074650 tx=0x7fce4800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bc865c0943ac7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce55600700 1 -- 172.21.15.3:0/2004580644 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800dd90 con 0x7fce5815d5b0 2024-09-13T16:52:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce55600700 1 -- 172.21.15.3:0/2004580644 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48010470 con 0x7fce5815d5b0 2024-09-13T16:52:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce55600700 1 -- 172.21.15.3:0/2004580644 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48010800 con 0x7fce5815d5b0 2024-09-13T16:52:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce5d600700 1 -- 172.21.15.3:0/2004580644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce5815d5b0 msgr2=0x7fce5806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce5d600700 1 --2- 172.21.15.3:0/2004580644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce5815d5b0 0x7fce5806d4b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fce58074650 tx=0x7fce4800be90 comp rx=0 tx=0).stop 2024-09-13T16:52:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce5d600700 1 -- 172.21.15.3:0/2004580644 shutdown_connections 2024-09-13T16:52:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce5d600700 1 --2- 172.21.15.3:0/2004580644 >> v2:172.21.15.157:3300/0 conn(0x7fce580cc8f0 0x7fce5815d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce5d600700 1 --2- 172.21.15.3:0/2004580644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce5815d5b0 0x7fce5806d4b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce5d600700 1 -- 172.21.15.3:0/2004580644 >> 172.21.15.3:0/2004580644 conn(0x7fce580b07a0 msgr2=0x7fce580b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:16.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce5d600700 1 -- 172.21.15.3:0/2004580644 shutdown_connections 2024-09-13T16:52:16.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.936+0000 7fce5d600700 1 -- 172.21.15.3:0/2004580644 wait complete. 2024-09-13T16:52:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.940+0000 7fce5d600700 1 Processor -- start 2024-09-13T16:52:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.940+0000 7fce5d600700 1 -- start start 2024-09-13T16:52:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.940+0000 7fce5d600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce580cc8f0 0x7fce58149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.940+0000 7fce5d600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58159fc0 0x7fce580b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.940+0000 7fce5d600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce5815d5b0 0x7fce581528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.940+0000 7fce5d600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58149980 con 0x7fce58159fc0 2024-09-13T16:52:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.940+0000 7fce5d600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580b2990 con 0x7fce580cc8f0 2024-09-13T16:52:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.940+0000 7fce5d600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce580b2c70 con 0x7fce5815d5b0 2024-09-13T16:52:16.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58159fc0 0x7fce580b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58159fc0 0x7fce580b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56030/0 (socket says 172.21.15.3:56030) 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 -- 172.21.15.3:0/1711700568 learned_addr learned my addr 172.21.15.3:0/1711700568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce57400700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce5815d5b0 0x7fce581528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56a00700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce580cc8f0 0x7fce58149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 -- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce5815d5b0 msgr2=0x7fce581528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce5815d5b0 0x7fce581528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 -- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce580cc8f0 msgr2=0x7fce58149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce580cc8f0 0x7fce58149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 -- 172.21.15.3:0/1711700568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4800b810 con 0x7fce58159fc0 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce57400700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce5815d5b0 0x7fce581528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56a00700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce580cc8f0 0x7fce58149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce56000700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58159fc0 0x7fce580b2090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fce4400e810 tx=0x7fce4400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce4ea00700 1 -- 172.21.15.3:0/1711700568 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44004230 con 0x7fce58159fc0 2024-09-13T16:52:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce4ea00700 1 -- 172.21.15.3:0/1711700568 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce44004390 con 0x7fce58159fc0 2024-09-13T16:52:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce580b2f50 con 0x7fce58159fc0 2024-09-13T16:52:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce4ea00700 1 -- 172.21.15.3:0/1711700568 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440036c0 con 0x7fce58159fc0 2024-09-13T16:52:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce580b3510 con 0x7fce58159fc0 2024-09-13T16:52:16.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.944+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5815d990 con 0x7fce58159fc0 2024-09-13T16:52:16.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.948+0000 7fce4ea00700 1 -- 172.21.15.3:0/1711700568 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce44003840 con 0x7fce58159fc0 2024-09-13T16:52:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.948+0000 7fce4ea00700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce38066730 0x7fce38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.948+0000 7fce4ea00700 1 -- 172.21.15.3:0/1711700568 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fce44083e20 con 0x7fce58159fc0 2024-09-13T16:52:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.952+0000 7fce56a00700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce38066730 0x7fce38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.952+0000 7fce56a00700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce38066730 0x7fce38068bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce4800bfd0 tx=0x7fce48013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:16.956+0000 7fce4ea00700 1 -- 172.21.15.3:0/1711700568 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce44050f60 con 0x7fce58159fc0 2024-09-13T16:52:17.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.200+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce580ccf60 con 0x7fce58159fc0 2024-09-13T16:52:17.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.200+0000 7fce4ea00700 1 -- 172.21.15.3:0/1711700568 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fce44020090 con 0x7fce58159fc0 2024-09-13T16:52:17.209 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:17.209 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:15.362813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5569":{"gid":5569,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/1311223400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1311223400},{"type":"v1","addr":"172.21.15.3:6835","nonce":1311223400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:17.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce38066730 msgr2=0x7fce38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce38066730 0x7fce38068bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce4800bfd0 tx=0x7fce48013000 comp rx=0 tx=0).stop 2024-09-13T16:52:17.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58159fc0 msgr2=0x7fce580b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58159fc0 0x7fce580b2090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fce4400e810 tx=0x7fce4400ebd0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 shutdown_connections 2024-09-13T16:52:17.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fce38066730 0x7fce38068bf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce5815d5b0 0x7fce581528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58159fc0 0x7fce580b2090 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 --2- 172.21.15.3:0/1711700568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce580cc8f0 0x7fce58149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 >> 172.21.15.3:0/1711700568 conn(0x7fce580b07a0 msgr2=0x7fce58070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 shutdown_connections 2024-09-13T16:52:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.204+0000 7fce5d600700 1 -- 172.21.15.3:0/1711700568 wait complete. 2024-09-13T16:52:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-09-13T16:52:17.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:52:03.049167+0000', 'modified': '2024-09-13T16:52:15.362813+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5569': {'gid': 5569, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.3:6835/1311223400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6832', 'nonce': 1311223400}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1311223400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:52:17.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:52:17.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T16:52:17.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d5d000700 1 Processor -- start 2024-09-13T16:52:17.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d5d000700 1 -- start start 2024-09-13T16:52:17.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d5d000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d580cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d5d000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d580d14e0 con 0x7f7d580cfbe0 2024-09-13T16:52:17.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d5d000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d580d17c0 con 0x7f7d5815cea0 2024-09-13T16:52:17.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d5d000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5815d280 con 0x7f7d580c7ef0 2024-09-13T16:52:17.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d56a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7d580cfbe0 0x7f7d580cd7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49936/0 (socket says 172.21.15.3:49936) 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d56a00700 1 -- 172.21.15.3:0/3177067981 learned_addr learned my addr 172.21.15.3:0/3177067981 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.812+0000 7f7d57e00700 1 --2- 172.21.15.3:0/3177067981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d580cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d56000700 1 -- 172.21.15.3:0/3177067981 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539289320 0 0) 0x7f7d580d17c0 con 0x7f7d5815cea0 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d56000700 1 -- 172.21.15.3:0/3177067981 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d3c0035d0 con 0x7f7d5815cea0 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d56000700 1 -- 172.21.15.3:0/3177067981 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258909464 0 0) 0x7f7d580d14e0 con 0x7f7d580cfbe0 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d56000700 1 -- 172.21.15.3:0/3177067981 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d580d17c0 con 0x7f7d580cfbe0 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d57e00700 1 -- 172.21.15.3:0/3177067981 >> v1:172.21.15.157:6789/0 conn(0x7f7d5815cea0 legacy=0x7f7d580cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d57e00700 1 -- 172.21.15.3:0/3177067981 >> v1:172.21.15.3:6789/0 conn(0x7f7d580cfbe0 legacy=0x7f7d580cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d57e00700 1 -- 172.21.15.3:0/3177067981 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580d1aa0 con 0x7f7d580c7ef0 2024-09-13T16:52:17.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d57e00700 1 --2- 172.21.15.3:0/3177067981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d580cdeb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009dc0 tx=0x7f7d5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48da387e39fe1fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:17.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d56000700 1 -- 172.21.15.3:0/3177067981 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000e070 con 0x7f7d580c7ef0 2024-09-13T16:52:17.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d56000700 1 -- 172.21.15.3:0/3177067981 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5000bb20 con 0x7f7d580c7ef0 2024-09-13T16:52:17.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d56000700 1 -- 172.21.15.3:0/3177067981 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000be90 con 0x7f7d580c7ef0 2024-09-13T16:52:17.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d5d000700 1 -- 172.21.15.3:0/3177067981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 msgr2=0x7f7d580cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d5d000700 1 --2- 172.21.15.3:0/3177067981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d580cdeb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009dc0 tx=0x7f7d5000b330 comp rx=0 tx=0).stop 2024-09-13T16:52:17.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d5d000700 1 -- 172.21.15.3:0/3177067981 shutdown_connections 2024-09-13T16:52:17.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d5d000700 1 --2- 172.21.15.3:0/3177067981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d580cdeb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d5d000700 1 -- 172.21.15.3:0/3177067981 >> 172.21.15.3:0/3177067981 conn(0x7f7d580d2b90 msgr2=0x7f7d580d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:17.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d5d000700 1 -- 172.21.15.3:0/3177067981 shutdown_connections 2024-09-13T16:52:17.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.816+0000 7f7d5d000700 1 -- 172.21.15.3:0/3177067981 wait complete. 2024-09-13T16:52:17.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.820+0000 7f7d5d000700 1 Processor -- start 2024-09-13T16:52:17.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.820+0000 7f7d5d000700 1 -- start start 2024-09-13T16:52:17.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d5d000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d58150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d5d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d580cfbe0 0x7f7d58150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d5d000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5815cea0 0x7f7d5806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d5d000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5815d630 con 0x7f7d5815cea0 2024-09-13T16:52:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d5d000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58157640 con 0x7f7d580cfbe0 2024-09-13T16:52:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d5d000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58150860 con 0x7f7d580c7ef0 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5815cea0 0x7f7d5806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5815cea0 0x7f7d5806bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56046/0 (socket says 172.21.15.3:56046) 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 -- 172.21.15.3:0/2999191590 learned_addr learned my addr 172.21.15.3:0/2999191590 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57400700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d58150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d56a00700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d580cfbe0 0x7f7d58150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 -- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 msgr2=0x7f7d58150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d58150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 -- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d580cfbe0 msgr2=0x7f7d58150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d580cfbe0 0x7f7d58150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 -- 172.21.15.3:0/2999191590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d50009a70 con 0x7f7d5815cea0 2024-09-13T16:52:17.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d57e00700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5815cea0 0x7f7d5806bdb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d50013000 tx=0x7f7d50004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:17.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d4f400700 1 -- 172.21.15.3:0/2999191590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50004880 con 0x7f7d5815cea0 2024-09-13T16:52:17.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d4f400700 1 -- 172.21.15.3:0/2999191590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50020070 con 0x7f7d5815cea0 2024-09-13T16:52:17.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d5d000700 1 -- 172.21.15.3:0/2999191590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d58156da0 con 0x7f7d5815cea0 2024-09-13T16:52:17.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d4f400700 1 -- 172.21.15.3:0/2999191590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50027680 con 0x7f7d5815cea0 2024-09-13T16:52:17.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.824+0000 7f7d5d000700 1 -- 172.21.15.3:0/2999191590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d58157360 con 0x7f7d5815cea0 2024-09-13T16:52:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.828+0000 7f7d4f400700 1 -- 172.21.15.3:0/2999191590 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d50012430 con 0x7f7d5815cea0 2024-09-13T16:52:17.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d4f400700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d38066730 0x7f7d38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d4f400700 1 -- 172.21.15.3:0/2999191590 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7d50022070 con 0x7f7d5815cea0 2024-09-13T16:52:17.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d57400700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d38066730 0x7f7d38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d61aaf740 1 Processor -- start 2024-09-13T16:52:17.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d61aaf740 1 -- start start 2024-09-13T16:52:17.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d57400700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d38066730 0x7f7d38068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7d48001010 tx=0x7f7d48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:17.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38b99890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 0x38b99dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x38af8d80 con 0x38bd0da0 2024-09-13T16:52:17.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x38af6510 con 0x38bafd00 2024-09-13T16:52:17.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x38af1e00 con 0x38bd16c0 2024-09-13T16:52:17.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56056/0 (socket says 172.21.15.3:56056) 2024-09-13T16:52:17.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d57400700 1 -- 172.21.15.3:0/4031468465 learned_addr learned my addr 172.21.15.3:0/4031468465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:17.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d57e00700 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 0x38b99dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.832+0000 7f7d56a00700 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38b99890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d57400700 1 -- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 msgr2=0x38b99dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d57400700 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 0x38b99dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d57400700 1 -- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 msgr2=0x38b99890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d57400700 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38b99890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d57400700 1 -- 172.21.15.3:0/4031468465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x38964960 con 0x38bd0da0 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d56a00700 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38b99890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d57400700 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd1180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800f690 tx=0x7f7d48012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b32086676fb261 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d43e00700 1 -- 172.21.15.3:0/4031468465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800fe10 con 0x38bd0da0 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d43e00700 1 -- 172.21.15.3:0/4031468465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d48014450 con 0x38bd0da0 2024-09-13T16:52:17.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d43e00700 1 -- 172.21.15.3:0/4031468465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d480147c0 con 0x38bd0da0 2024-09-13T16:52:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4031468465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 msgr2=0x38bd1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd1180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800f690 tx=0x7f7d48012000 comp rx=0 tx=0).stop 2024-09-13T16:52:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4031468465 shutdown_connections 2024-09-13T16:52:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 0x38b99dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38b99890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/4031468465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd1180 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4031468465 >> 172.21.15.3:0/4031468465 conn(0x389194d0 msgr2=0x38bb2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4031468465 shutdown_connections 2024-09-13T16:52:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4031468465 wait complete. 2024-09-13T16:52:17.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 -- start start 2024-09-13T16:52:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.836+0000 7f7d61aaf740 1 Processor -- start 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38bd1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 0x38bd9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x38aaf730 con 0x38bd0da0 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x38ab55b0 con 0x38bafd00 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3899daa0 con 0x38bd16c0 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 10 client.0 _ll_register_callbacks cb 0x38aeb610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56058/0 (socket says 172.21.15.3:56058) 2024-09-13T16:52:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 -- 172.21.15.3:0/2991487131 learned_addr learned my addr 172.21.15.3:0/2991487131 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d57e00700 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 0x38bd9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d57400700 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38bd1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 -- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 msgr2=0x38bd9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 0x38bd9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 -- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 msgr2=0x38bd1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38bd1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 -- 172.21.15.3:0/2991487131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c0035d0 con 0x38bd0da0 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d57400700 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38bd1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d56a00700 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd7130 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d44007690 tx=0x7f7d44001350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b7108422610433 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d40c00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d40c00700 1 -- 172.21.15.3:0/2991487131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44001c30 con 0x38bd0da0 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d40c00700 1 -- 172.21.15.3:0/2991487131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7d44015470 con 0x38bd0da0 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 -- 172.21.15.3:0/2991487131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x389261e0 con 0x38bd0da0 2024-09-13T16:52:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d40c00700 1 -- 172.21.15.3:0/2991487131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d44015630 con 0x38bd0da0 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 10 client.5622 fetch_fsmap learned FSMap version 22 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 -- 172.21.15.3:0/2991487131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x38ab8220 con 0x38bd0da0 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d40c00700 1 -- 172.21.15.3:0/2991487131 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x389261e0 con 0x38bd0da0 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d40c00700 1 -- 172.21.15.3:0/2991487131 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7d44015790 con 0x38bd0da0 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 10 client.5622 fetch_fsmap finished waiting for FSMap version 22 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 10 client.5622 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 20 client.5622 populate_metadata read hostname 'smithi003' 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] conn(0x38b89450 0x38b8a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 4 client.5622 mds_command: new command op to 5569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.840+0000 7f7d61aaf740 1 -- 172.21.15.3:0/2991487131 --> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x38970460 con 0x38b89450 2024-09-13T16:52:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d57400700 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] conn(0x38b89450 0x38b8a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d57400700 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] conn(0x38b89450 0x38b8a730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f70786404372a5 server_cookie=8940b9844ba6847c in_seq=0 out_seq=0 2024-09-13T16:52:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d40c00700 10 client.5622 ms_handle_connect on v2:172.21.15.3:6832/1311223400 2024-09-13T16:52:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d40c00700 1 -- 172.21.15.3:0/2991487131 <== mds.0 v2:172.21.15.3:6832/1311223400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x38970460 con 0x38b89450 2024-09-13T16:52:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d40c00700 10 client.5622 handle_command_reply: tid=0 2024-09-13T16:52:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 client.5622 shutdown 2024-09-13T16:52:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 -- 172.21.15.3:0/2991487131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 msgr2=0x38bd7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd7130 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d44007690 tx=0x7f7d44001350 comp rx=0 tx=0).stop 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 -- 172.21.15.3:0/2991487131 shutdown_connections 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] conn(0x38b89450 0x38b8a730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd16c0 0x38bd9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d40c00700 0 client.5622 ms_handle_reset on v2:172.21.15.3:6832/1311223400 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd0da0 0x38bd7130 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/2991487131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38bd1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 -- 172.21.15.3:0/2991487131 >> 172.21.15.3:0/2991487131 conn(0x389194d0 msgr2=0x38bac780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 -- 172.21.15.3:0/2991487131 shutdown_connections 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 1 -- 172.21.15.3:0/2991487131 wait complete. 2024-09-13T16:52:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.844+0000 7f7d61aaf740 20 client.5622 trim_cache size 0 max 16384 2024-09-13T16:52:17.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d61aaf740 1 Processor -- start 2024-09-13T16:52:17.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d61aaf740 1 -- start start 2024-09-13T16:52:17.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38baf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 0x38bd1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd16c0 0x38bd1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x38b8adb0 con 0x38bd16c0 2024-09-13T16:52:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x38a9a020 con 0x38bafd00 2024-09-13T16:52:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x38af1610 con 0x38bd0da0 2024-09-13T16:52:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d57e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd16c0 0x38bd1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d57e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd16c0 0x38bd1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56060/0 (socket says 172.21.15.3:56060) 2024-09-13T16:52:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d56a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 0x38bd1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d57e00700 1 -- 172.21.15.3:0/4140173755 learned_addr learned my addr 172.21.15.3:0/4140173755 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.868+0000 7f7d57400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38baf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d57e00700 1 -- 172.21.15.3:0/4140173755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 msgr2=0x38bd1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d57e00700 1 --2- 172.21.15.3:0/4140173755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 0x38bd1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d57e00700 1 -- 172.21.15.3:0/4140173755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 msgr2=0x38baf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d57e00700 1 --2- 172.21.15.3:0/4140173755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38baf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d57e00700 1 -- 172.21.15.3:0/4140173755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x38a9e450 con 0x38bd16c0 2024-09-13T16:52:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d57e00700 1 --2- 172.21.15.3:0/4140173755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd16c0 0x38bd1fe0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009d90 tx=0x7f7d50052f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7802802df888768 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d4cc00700 1 -- 172.21.15.3:0/4140173755 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50003e30 con 0x38bd16c0 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d4cc00700 1 -- 172.21.15.3:0/4140173755 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d500572d0 con 0x38bd16c0 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d4cc00700 1 -- 172.21.15.3:0/4140173755 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50056bd0 con 0x38bd16c0 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4140173755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd16c0 msgr2=0x38bd1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/4140173755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd16c0 0x38bd1fe0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009d90 tx=0x7f7d50052f10 comp rx=0 tx=0).stop 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4140173755 shutdown_connections 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/4140173755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bd16c0 0x38bd1fe0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/4140173755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 0x38bd1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/4140173755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bafd00 0x38baf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4140173755 >> 172.21.15.3:0/4140173755 conn(0x389194d0 msgr2=0x38b87f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4140173755 shutdown_connections 2024-09-13T16:52:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 -- 172.21.15.3:0/4140173755 wait complete. 2024-09-13T16:52:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 -- start start 2024-09-13T16:52:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.872+0000 7f7d61aaf740 1 Processor -- start 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bafd00 0x38c3dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 0x38bf9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bd16c0 0x38bf9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x38b8bf80 con 0x38bafd00 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x38b8a490 con 0x38bd16c0 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x38b8a260 con 0x38bd0da0 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 10 client.0 _ll_register_callbacks cb 0x38b48710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bafd00 0x38c3dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bafd00 0x38c3dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56064/0 (socket says 172.21.15.3:56064) 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 -- 172.21.15.3:0/1720044861 learned_addr learned my addr 172.21.15.3:0/1720044861 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57e00700 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bd16c0 0x38bf9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d56a00700 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 0x38bf9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 -- 172.21.15.3:0/1720044861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 msgr2=0x38bf9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 0x38bf9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 -- 172.21.15.3:0/1720044861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bd16c0 msgr2=0x38bf9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bd16c0 0x38bf9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 -- 172.21.15.3:0/1720044861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d58157360 con 0x38bafd00 2024-09-13T16:52:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d57400700 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bafd00 0x38c3dd00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7d48023040 tx=0x7f7d48001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da19d82645307ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d40c00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:52:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d40c00700 1 -- 172.21.15.3:0/1720044861 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d48030070 con 0x38bafd00 2024-09-13T16:52:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d40c00700 1 -- 172.21.15.3:0/1720044861 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7d4801c040 con 0x38bafd00 2024-09-13T16:52:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 1 -- 172.21.15.3:0/1720044861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d48014450 con 0x38bafd00 2024-09-13T16:52:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d40c00700 1 -- 172.21.15.3:0/1720044861 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d48014640 con 0x38bafd00 2024-09-13T16:52:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 10 client.5628 fetch_fsmap learned FSMap version 22 2024-09-13T16:52:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d61aaf740 1 -- 172.21.15.3:0/1720044861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x38a92720 con 0x38bafd00 2024-09-13T16:52:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.876+0000 7f7d40c00700 1 -- 172.21.15.3:0/1720044861 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d48014450 con 0x38bafd00 2024-09-13T16:52:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d40c00700 1 -- 172.21.15.3:0/1720044861 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7d48031d10 con 0x38bafd00 2024-09-13T16:52:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 10 client.5628 fetch_fsmap finished waiting for FSMap version 22 2024-09-13T16:52:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 10 client.5628 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T16:52:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 20 client.5628 populate_metadata read hostname 'smithi003' 2024-09-13T16:52:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] conn(0x38bd4750 0x38bd4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 4 client.5628 mds_command: new command op to 5569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:52:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 -- 172.21.15.3:0/1720044861 --> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] -- command(tid 0: {"prefix": "status"}) v1 -- 0x389951e0 con 0x38bd4750 2024-09-13T16:52:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d56a00700 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] conn(0x38bd4750 0x38bd4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d40c00700 10 client.5628 ms_handle_connect on v2:172.21.15.3:6832/1311223400 2024-09-13T16:52:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d56a00700 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] conn(0x38bd4750 0x38bd4b30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a36a1934879ee26 server_cookie=2c164ce1a7ca4e34 in_seq=0 out_seq=0 2024-09-13T16:52:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d40c00700 1 -- 172.21.15.3:0/1720044861 <== mds.0 v2:172.21.15.3:6832/1311223400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x389951e0 con 0x38bd4750 2024-09-13T16:52:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d40c00700 10 client.5628 handle_command_reply: tid=0 2024-09-13T16:52:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 client.5628 shutdown 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 -- 172.21.15.3:0/1720044861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bafd00 msgr2=0x38c3dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bafd00 0x38c3dd00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7d48023040 tx=0x7f7d48001080 comp rx=0 tx=0).stop 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 -- 172.21.15.3:0/1720044861 shutdown_connections 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.3:6832/1311223400,v1:172.21.15.3:6835/1311223400] conn(0x38bd4750 0x38bd4b30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x38bd16c0 0x38bf9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x38bd0da0 0x38bf9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 --2- 172.21.15.3:0/1720044861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x38bafd00 0x38c3dd00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d40c00700 0 client.5628 ms_handle_reset on v2:172.21.15.3:6832/1311223400 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 -- 172.21.15.3:0/1720044861 >> 172.21.15.3:0/1720044861 conn(0x389194d0 msgr2=0x38b87f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 -- 172.21.15.3:0/1720044861 shutdown_connections 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 1 -- 172.21.15.3:0/1720044861 wait complete. 2024-09-13T16:52:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d61aaf740 20 client.5628 trim_cache size 0 max 16384 2024-09-13T16:52:17.888 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:52:17.888 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "61456116-a42d-4065-9bf5-a1daf0be594e", 2024-09-13T16:52:17.888 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-13T16:52:17.888 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5569, 2024-09-13T16:52:17.888 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-13T16:52:17.888 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-13T16:52:17.888 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-13T16:52:17.888 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 6.5460980270000002, 2024-09-13T16:52:17.889 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-09-13T16:52:17.889 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 46, 2024-09-13T16:52:17.889 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 45, 2024-09-13T16:52:17.889 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 6.5857396110000002 2024-09-13T16:52:17.889 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:52:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 -- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d38066730 msgr2=0x7f7d38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d38066730 0x7f7d38068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7d48001010 tx=0x7f7d48009040 comp rx=0 tx=0).stop 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 -- 172.21.15.3:0/2999191590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5815cea0 msgr2=0x7f7d5806bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5815cea0 0x7f7d5806bdb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d50013000 tx=0x7f7d50004af0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 -- 172.21.15.3:0/2999191590 shutdown_connections 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7d38066730 0x7f7d38068bf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5815cea0 0x7f7d5806bdb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d580cfbe0 0x7f7d58150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 --2- 172.21.15.3:0/2999191590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d580c7ef0 0x7f7d58150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.880+0000 7f7d5d000700 1 -- 172.21.15.3:0/2999191590 >> 172.21.15.3:0/2999191590 conn(0x7f7d580d2b90 msgr2=0x7f7d580ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.884+0000 7f7d5d000700 1 -- 172.21.15.3:0/2999191590 shutdown_connections 2024-09-13T16:52:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:17.884+0000 7f7d5d000700 1 -- 172.21.15.3:0/2999191590 wait complete. 2024-09-13T16:52:17.912 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:17.912 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:17.928 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:52:17.928 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:52:17.928 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:52:17.928 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/open_file0", '"'"'w'"'"') as f: 2024-09-13T16:52:17.928 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:52:17.928 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:52:17.929 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:52:17.929 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:52:17.929 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:52:17.976 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file0 2024-09-13T16:52:17.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:52:17.996 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file0': No such file or directory 2024-09-13T16:52:18.997 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file0 2024-09-13T16:52:19.015 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:52:19.015 INFO:tasks.cephfs.mount:File open_file0 became visible with size 7 from 0 after 1s 2024-09-13T16:52:19.016 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:19.016 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:19.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:19.076 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T16:52:19.177 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:52:19.177 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:52:19.177 INFO:teuthology.orchestra.run.smithi003.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286654 s, 36.6 MB/s 2024-09-13T16:52:19.178 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:19.179 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:19.195 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:52:19.195 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:52:19.195 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:52:19.195 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/open_file1", '"'"'w'"'"') as f: 2024-09-13T16:52:19.195 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:52:19.195 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:52:19.195 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:52:19.196 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:52:19.196 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:52:19.244 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file1 2024-09-13T16:52:19.263 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file1': No such file or directory 2024-09-13T16:52:19.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:52:20.264 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file1 2024-09-13T16:52:20.285 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:52:20.285 INFO:tasks.cephfs.mount:File open_file1 became visible with size 7 from 0 after 1s 2024-09-13T16:52:20.286 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:20.286 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:20.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:20.347 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T16:52:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:52:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:52:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336945 s, 31.1 MB/s 2024-09-13T16:52:20.456 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:20.457 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/open_file2", '"'"'w'"'"') as f: 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:52:20.471 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:52:20.516 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file2 2024-09-13T16:52:20.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:52:20.535 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file2': No such file or directory 2024-09-13T16:52:21.536 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file2 2024-09-13T16:52:21.555 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:52:21.556 INFO:tasks.cephfs.mount:File open_file2 became visible with size 7 from 0 after 1s 2024-09-13T16:52:21.556 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:21.556 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:21.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:21.616 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T16:52:21.720 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:52:21.720 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:52:21.721 INFO:teuthology.orchestra.run.smithi003.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317296 s, 33.0 MB/s 2024-09-13T16:52:21.722 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:21.722 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:21.738 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:52:21.738 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:52:21.738 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:52:21.738 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/open_file3", '"'"'w'"'"') as f: 2024-09-13T16:52:21.738 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:52:21.738 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:52:21.738 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:52:21.739 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:52:21.739 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:52:21.789 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file3 2024-09-13T16:52:21.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:52:21.809 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file3': No such file or directory 2024-09-13T16:52:22.811 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file3 2024-09-13T16:52:22.832 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:52:22.833 INFO:tasks.cephfs.mount:File open_file3 became visible with size 7 from 0 after 1s 2024-09-13T16:52:22.833 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:22.833 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:22.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:22.891 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T16:52:23.001 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:52:23.002 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:52:23.002 INFO:teuthology.orchestra.run.smithi003.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304286 s, 34.5 MB/s 2024-09-13T16:52:23.003 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:23.003 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/open_file4", '"'"'w'"'"') as f: 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:52:23.016 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:52:23.065 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file4 2024-09-13T16:52:23.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:52:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file4': No such file or directory 2024-09-13T16:52:24.086 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file4 2024-09-13T16:52:24.105 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:52:24.106 INFO:tasks.cephfs.mount:File open_file4 became visible with size 7 from 0 after 1s 2024-09-13T16:52:24.106 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:24.106 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:24.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:24.166 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T16:52:24.268 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:52:24.268 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:52:24.268 INFO:teuthology.orchestra.run.smithi003.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265228 s, 39.5 MB/s 2024-09-13T16:52:24.270 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:24.270 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:24.284 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:52:24.284 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:52:24.284 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:52:24.284 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/open_file5", '"'"'w'"'"') as f: 2024-09-13T16:52:24.284 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:52:24.284 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:52:24.284 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:52:24.285 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:52:24.285 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:52:24.333 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file5 2024-09-13T16:52:24.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:52:24.351 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file5': No such file or directory 2024-09-13T16:52:25.353 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file5 2024-09-13T16:52:25.373 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:52:25.373 INFO:tasks.cephfs.mount:File open_file5 became visible with size 7 from 0 after 1s 2024-09-13T16:52:25.373 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:25.373 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:25.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:25.433 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T16:52:25.528 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:52:25.528 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:52:25.529 INFO:teuthology.orchestra.run.smithi003.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227668 s, 46.1 MB/s 2024-09-13T16:52:25.530 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:25.530 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:25.550 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:52:25.550 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:52:25.550 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:52:25.550 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/open_file6", '"'"'w'"'"') as f: 2024-09-13T16:52:25.550 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:52:25.550 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:52:25.550 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:52:25.551 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:52:25.551 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:52:25.595 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file6 2024-09-13T16:52:25.611 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file6': No such file or directory 2024-09-13T16:52:25.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:52:26.612 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file6 2024-09-13T16:52:26.628 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:52:26.632 INFO:tasks.cephfs.mount:File open_file6 became visible with size 7 from 0 after 1s 2024-09-13T16:52:26.633 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:26.633 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:26.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:26.691 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T16:52:26.796 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:52:26.796 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:52:26.796 INFO:teuthology.orchestra.run.smithi003.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328409 s, 31.9 MB/s 2024-09-13T16:52:26.797 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:26.797 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:26.808 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:52:26.809 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:52:26.809 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:52:26.809 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/open_file7", '"'"'w'"'"') as f: 2024-09-13T16:52:26.809 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:52:26.809 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:52:26.809 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:52:26.809 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:52:26.809 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:52:26.857 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file7 2024-09-13T16:52:26.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:52:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file7': No such file or directory 2024-09-13T16:52:27.876 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file7 2024-09-13T16:52:27.895 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:52:27.895 INFO:tasks.cephfs.mount:File open_file7 became visible with size 7 from 0 after 1s 2024-09-13T16:52:27.896 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:27.896 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:27.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:27.958 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T16:52:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-13T16:52:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-13T16:52:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272337 s, 38.5 MB/s 2024-09-13T16:52:28.715 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:28.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:38.059 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.1 2024-09-13T16:52:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb04a429140 1 Processor -- start 2024-09-13T16:52:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb04a429140 1 -- start start 2024-09-13T16:52:38.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb04a429140 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6e6a510 0x5653e6e9cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:38.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb04a429140 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6f41330 0x5653e6eae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:38.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb04a429140 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e9d360 0x5653e6f2e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:38.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb04a429140 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e6dfd2f0 con 0x5653e6e9d360 2024-09-13T16:52:38.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb04a429140 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e6db98e0 con 0x5653e6f41330 2024-09-13T16:52:38.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb04a429140 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653e6db9580 con 0x5653e6e6a510 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e9d360 0x5653e6f2e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e9d360 0x5653e6f2e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43754/0 (socket says 172.21.15.3:43754) 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 -- 172.21.15.3:0/3044397746 learned_addr learned my addr 172.21.15.3:0/3044397746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb046c00700 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6f41330 0x5653e6eae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb047600700 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6e6a510 0x5653e6e9cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 -- 172.21.15.3:0/3044397746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6e6a510 msgr2=0x5653e6e9cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6e6a510 0x5653e6e9cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 -- 172.21.15.3:0/3044397746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6f41330 msgr2=0x5653e6eae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6f41330 0x5653e6eae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 -- 172.21.15.3:0/3044397746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb038005de0 con 0x5653e6e9d360 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb048000700 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e9d360 0x5653e6f2e450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb0400014a0 tx=0x7fb0400017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147f44200ca667ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb045800700 1 -- 172.21.15.3:0/3044397746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040012600 con 0x5653e6e9d360 2024-09-13T16:52:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb045800700 1 -- 172.21.15.3:0/3044397746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040012780 con 0x5653e6e9d360 2024-09-13T16:52:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.132+0000 7fb045800700 1 -- 172.21.15.3:0/3044397746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040012b30 con 0x5653e6e9d360 2024-09-13T16:52:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- 172.21.15.3:0/3044397746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e9d360 msgr2=0x5653e6f2e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e9d360 0x5653e6f2e450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb0400014a0 tx=0x7fb0400017b0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- 172.21.15.3:0/3044397746 shutdown_connections 2024-09-13T16:52:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e9d360 0x5653e6f2e450 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6f41330 0x5653e6eae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 --2- 172.21.15.3:0/3044397746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6e6a510 0x5653e6e9cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- 172.21.15.3:0/3044397746 >> 172.21.15.3:0/3044397746 conn(0x5653e6e6ace0 msgr2=0x5653e6e9c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- 172.21.15.3:0/3044397746 shutdown_connections 2024-09-13T16:52:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- 172.21.15.3:0/3044397746 wait complete. 2024-09-13T16:52:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 Processor -- start 2024-09-13T16:52:38.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- start start 2024-09-13T16:52:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e6a510 0x5653e6e90ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6e9d360 0x5653e6eaf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6f41330 0x5653e6e9c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e6d8f8d0 con 0x5653e6e6a510 2024-09-13T16:52:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e6e95cb0 con 0x5653e6e9d360 2024-09-13T16:52:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb04a429140 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653e6e95f50 con 0x5653e6f41330 2024-09-13T16:52:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb047600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e6a510 0x5653e6e90ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb047600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e6a510 0x5653e6e90ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43766/0 (socket says 172.21.15.3:43766) 2024-09-13T16:52:38.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb047600700 1 -- 172.21.15.3:0/3626883134 learned_addr learned my addr 172.21.15.3:0/3626883134 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.136+0000 7fb048000700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6f41330 0x5653e6e9c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb046c00700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6e9d360 0x5653e6eaf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb047600700 1 -- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6f41330 msgr2=0x5653e6e9c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb047600700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5653e6f41330 0x5653e6e9c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb047600700 1 -- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6e9d360 msgr2=0x5653e6eaf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb047600700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653e6e9d360 0x5653e6eaf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb047600700 1 -- 172.21.15.3:0/3626883134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653e6d9abb0 con 0x5653e6e6a510 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb047600700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e6a510 0x5653e6e90ef0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb038011000 tx=0x7fb038002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb03fe00700 1 -- 172.21.15.3:0/3626883134 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038005860 con 0x5653e6e6a510 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb03fe00700 1 -- 172.21.15.3:0/3626883134 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb038005540 con 0x5653e6e6a510 2024-09-13T16:52:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653e6da7b00 con 0x5653e6e6a510 2024-09-13T16:52:38.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb03fe00700 1 -- 172.21.15.3:0/3626883134 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03801d850 con 0x5653e6e6a510 2024-09-13T16:52:38.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.140+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653e6d9af40 con 0x5653e6e6a510 2024-09-13T16:52:38.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.144+0000 7fb03fe00700 1 -- 172.21.15.3:0/3626883134 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb03801d9b0 con 0x5653e6e6a510 2024-09-13T16:52:38.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.144+0000 7fb03fe00700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb028065090 0x7fb028067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:38.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.144+0000 7fb03fe00700 1 -- 172.21.15.3:0/3626883134 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb038084500 con 0x5653e6e6a510 2024-09-13T16:52:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.144+0000 7fb04a429140 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x5653e6e9cc40 0x5653e6f29590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.144+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 --> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] -- osd_op(unknown.0.0:1 5.13 5:c9647462:::mds0_openfiles.1:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) v8 -- 0x5653e6f2b260 con 0x5653e6e9cc40 2024-09-13T16:52:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.144+0000 7fb046c00700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb028065090 0x7fb028067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.144+0000 7fb048000700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x5653e6e9cc40 0x5653e6f29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:38.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.144+0000 7fb046c00700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb028065090 0x7fb028067550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb034009800 tx=0x7fb03400c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:38.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb048000700 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x5653e6e9cc40 0x5653e6f29590 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:38.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb048000700 1 -- 172.21.15.3:0/3626883134 <== osd.3 v2:172.21.15.3:6813/1788023973 1 ==== osd_op_reply(1 mds0_openfiles.1 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x7fb040012480 con 0x5653e6e9cc40 2024-09-13T16:52:38.152 INFO:teuthology.orchestra.run.smithi003.stdout:cephfs_metadata/mds0_openfiles.1 mtime 2024-09-13T16:52:31.000000+0000, size 0 2024-09-13T16:52:38.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x5653e6e9cc40 msgr2=0x5653e6f29590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:38.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x5653e6e9cc40 0x5653e6f29590 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb028065090 msgr2=0x7fb028067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb028065090 0x7fb028067550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb034009800 tx=0x7fb03400c040 comp rx=0 tx=0).stop 2024-09-13T16:52:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e6a510 msgr2=0x5653e6e90ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 --2- 172.21.15.3:0/3626883134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e6e6a510 0x5653e6e90ef0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb038011000 tx=0x7fb038002fb0 comp rx=0 tx=0).stop 2024-09-13T16:52:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb048000700 1 -- 172.21.15.3:0/3626883134 reap_dead start 2024-09-13T16:52:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 shutdown_connections 2024-09-13T16:52:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 >> 172.21.15.3:0/3626883134 conn(0x5653e6e6ace0 msgr2=0x5653e6eb0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 shutdown_connections 2024-09-13T16:52:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:38.148+0000 7fb04a429140 1 -- 172.21.15.3:0/3626883134 wait complete. 2024-09-13T16:52:38.176 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T16:52:38.176 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T16:52:38.272 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T16:52:38.314 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T16:52:38.315 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T16:52:38.338 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T16:52:38.395 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T16:52:38.456 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T16:52:38.515 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T16:52:38.579 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T16:52:38.643 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T16:52:38.706 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:38.706 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:38.768 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:38.768 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:38.829 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:39.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.080+0000 7fb463e00700 1 Processor -- start 2024-09-13T16:52:39.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.080+0000 7fb463e00700 1 -- start start 2024-09-13T16:52:39.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb463e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb464165770 0x7fb464164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb463e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb464165110 0x7fb464070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb463e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46406ba80 con 0x7fb464160f30 2024-09-13T16:52:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb463e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46406fad0 con 0x7fb464165770 2024-09-13T16:52:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb463e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb464165550 con 0x7fb464165110 2024-09-13T16:52:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb464165770 0x7fb464164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb464165770 0x7fb464164bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43780/0 (socket says 172.21.15.3:43780) 2024-09-13T16:52:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb464160f30 0x7fb46406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34780/0 (socket says 172.21.15.3:34780) 2024-09-13T16:52:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462000700 1 -- 172.21.15.3:0/4232842797 learned_addr learned my addr 172.21.15.3:0/4232842797 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb463400700 1 --2- 172.21.15.3:0/4232842797 >> v2:172.21.15.157:3300/0 conn(0x7fb464165110 0x7fb464070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462000700 1 -- 172.21.15.3:0/4232842797 >> v2:172.21.15.157:3300/0 conn(0x7fb464165110 msgr2=0x7fb464070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462000700 1 --2- 172.21.15.3:0/4232842797 >> v2:172.21.15.157:3300/0 conn(0x7fb464165110 0x7fb464070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462000700 1 -- 172.21.15.3:0/4232842797 >> v1:172.21.15.3:6789/0 conn(0x7fb464160f30 legacy=0x7fb46406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462000700 1 -- 172.21.15.3:0/4232842797 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46406bcd0 con 0x7fb464165770 2024-09-13T16:52:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb463400700 1 --2- 172.21.15.3:0/4232842797 >> v2:172.21.15.157:3300/0 conn(0x7fb464165110 0x7fb464070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb462000700 1 --2- 172.21.15.3:0/4232842797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165770 0x7fb464164bd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb4640719c0 tx=0x7fb450009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912042ba757b19d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.084+0000 7fb461600700 1 -- 172.21.15.3:0/4232842797 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45000b8a0 con 0x7fb464165770 2024-09-13T16:52:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb461600700 1 -- 172.21.15.3:0/4232842797 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45000ba00 con 0x7fb464165770 2024-09-13T16:52:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb461600700 1 -- 172.21.15.3:0/4232842797 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45000bd90 con 0x7fb464165770 2024-09-13T16:52:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb463e00700 1 -- 172.21.15.3:0/4232842797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165770 msgr2=0x7fb464164bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb463e00700 1 --2- 172.21.15.3:0/4232842797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165770 0x7fb464164bd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb4640719c0 tx=0x7fb450009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb463e00700 1 -- 172.21.15.3:0/4232842797 shutdown_connections 2024-09-13T16:52:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb463e00700 1 --2- 172.21.15.3:0/4232842797 >> v2:172.21.15.157:3300/0 conn(0x7fb464165110 0x7fb464070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb463e00700 1 --2- 172.21.15.3:0/4232842797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165770 0x7fb464164bd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb463e00700 1 -- 172.21.15.3:0/4232842797 >> 172.21.15.3:0/4232842797 conn(0x7fb46406a2f0 msgr2=0x7fb46406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb463e00700 1 -- 172.21.15.3:0/4232842797 shutdown_connections 2024-09-13T16:52:39.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.088+0000 7fb463e00700 1 -- 172.21.15.3:0/4232842797 wait complete. 2024-09-13T16:52:39.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463e00700 1 Processor -- start 2024-09-13T16:52:39.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463e00700 1 -- start start 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb464160f30 0x7fb464150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165110 0x7fb46414cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb464165770 0x7fb464152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641522d0 con 0x7fb464165110 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641513b0 con 0x7fb464160f30 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb464151650 con 0x7fb464165770 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb462000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165110 0x7fb46414cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb462000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165110 0x7fb46414cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43792/0 (socket says 172.21.15.3:43792) 2024-09-13T16:52:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb462000700 1 -- 172.21.15.3:0/2874246824 learned_addr learned my addr 172.21.15.3:0/2874246824 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb462a00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb464160f30 0x7fb464150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.092+0000 7fb463400700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb464165770 0x7fb464152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb462000700 1 -- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb464165770 msgr2=0x7fb464152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb462000700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb464165770 0x7fb464152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb462000700 1 -- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb464160f30 msgr2=0x7fb464150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb462000700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb464160f30 0x7fb464150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb462000700 1 -- 172.21.15.3:0/2874246824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450009570 con 0x7fb464165110 2024-09-13T16:52:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb463400700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb464165770 0x7fb464152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb462000700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165110 0x7fb46414cc80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb450000c00 tx=0x7fb450004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb45aa00700 1 -- 172.21.15.3:0/2874246824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450003ad0 con 0x7fb464165110 2024-09-13T16:52:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb45aa00700 1 -- 172.21.15.3:0/2874246824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45001e070 con 0x7fb464165110 2024-09-13T16:52:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb464151930 con 0x7fb464165110 2024-09-13T16:52:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb45aa00700 1 -- 172.21.15.3:0/2874246824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450003e90 con 0x7fb464165110 2024-09-13T16:52:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb464151ef0 con 0x7fb464165110 2024-09-13T16:52:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.096+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb464164bd0 con 0x7fb464165110 2024-09-13T16:52:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.100+0000 7fb45aa00700 1 -- 172.21.15.3:0/2874246824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb45001a420 con 0x7fb464165110 2024-09-13T16:52:39.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.100+0000 7fb45aa00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb444066780 0x7fb444068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.100+0000 7fb45aa00700 1 -- 172.21.15.3:0/2874246824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb450020070 con 0x7fb464165110 2024-09-13T16:52:39.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.100+0000 7fb462a00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb444066780 0x7fb444068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.104+0000 7fb462a00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb444066780 0x7fb444068c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb464150710 tx=0x7fb454007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.104+0000 7fb45aa00700 1 -- 172.21.15.3:0/2874246824 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb450052d20 con 0x7fb464165110 2024-09-13T16:52:39.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.296+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb464164940 con 0x7fb464165110 2024-09-13T16:52:39.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.296+0000 7fb45aa00700 1 -- 172.21.15.3:0/2874246824 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16900 (secure 0 0 0) 0x7fb450056340 con 0x7fb464165110 2024-09-13T16:52:39.303 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:39.304 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":46,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:15.362510+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:01.009263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:02.059017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/750657669":"2024-09-14T16:52:11.310659+0000","172.21.15.3:6834/750657669":"2024-09-14T16:52:11.310659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:39.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb444066780 msgr2=0x7fb444068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb444066780 0x7fb444068c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb464150710 tx=0x7fb454007f30 comp rx=0 tx=0).stop 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165110 msgr2=0x7fb46414cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165110 0x7fb46414cc80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb450000c00 tx=0x7fb450004770 comp rx=0 tx=0).stop 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 shutdown_connections 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb444066780 0x7fb444068c40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb464165770 0x7fb464152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb464165110 0x7fb46414cc80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 --2- 172.21.15.3:0/2874246824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb464160f30 0x7fb464150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 >> 172.21.15.3:0/2874246824 conn(0x7fb46406a2f0 msgr2=0x7fb4640709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 shutdown_connections 2024-09-13T16:52:39.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.300+0000 7fb463e00700 1 -- 172.21.15.3:0/2874246824 wait complete. 2024-09-13T16:52:39.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:52:01.009263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:52:02.059017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:52:39.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:52:39.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.524+0000 7f0d5e200700 1 Processor -- start 2024-09-13T16:52:39.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.524+0000 7f0d5e200700 1 -- start start 2024-09-13T16:52:39.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d5e200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d58071e60 0x7f0d5815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d5e200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580ce470 0x7f0d580b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d5e200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d580cd130 con 0x7f0d580b29d0 2024-09-13T16:52:39.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d5e200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d580b2e40 con 0x7f0d58071e60 2024-09-13T16:52:39.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d5e200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d580cd3d0 con 0x7f0d580ce470 2024-09-13T16:52:39.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d57e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0d580b29d0 0x7f0d58160940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34788/0 (socket says 172.21.15.3:34788) 2024-09-13T16:52:39.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d57e00700 1 -- 172.21.15.3:0/2158355657 learned_addr learned my addr 172.21.15.3:0/2158355657 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:39.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d57400700 1 --2- 172.21.15.3:0/2158355657 >> v2:172.21.15.157:3300/0 conn(0x7f0d58071e60 0x7f0d5815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d56a00700 1 --2- 172.21.15.3:0/2158355657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580ce470 0x7f0d580b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d56000700 1 -- 172.21.15.3:0/2158355657 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303367737 0 0) 0x7f0d580cd130 con 0x7f0d580b29d0 2024-09-13T16:52:39.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d56000700 1 -- 172.21.15.3:0/2158355657 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d3c0035d0 con 0x7f0d580b29d0 2024-09-13T16:52:39.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d57400700 1 -- 172.21.15.3:0/2158355657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580ce470 msgr2=0x7f0d580b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:39.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d57400700 1 --2- 172.21.15.3:0/2158355657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580ce470 0x7f0d580b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d57400700 1 -- 172.21.15.3:0/2158355657 >> v1:172.21.15.3:6789/0 conn(0x7f0d580b29d0 legacy=0x7f0d58160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d57400700 1 -- 172.21.15.3:0/2158355657 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d580cd6b0 con 0x7f0d58071e60 2024-09-13T16:52:39.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d57400700 1 --2- 172.21.15.3:0/2158355657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d58071e60 0x7f0d5815dd70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009850 tx=0x7f0d48009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2001bf5625a2dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d56000700 1 -- 172.21.15.3:0/2158355657 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48011830 con 0x7f0d58071e60 2024-09-13T16:52:39.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d56000700 1 -- 172.21.15.3:0/2158355657 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d48011990 con 0x7f0d58071e60 2024-09-13T16:52:39.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.528+0000 7f0d56000700 1 -- 172.21.15.3:0/2158355657 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48011d40 con 0x7f0d58071e60 2024-09-13T16:52:39.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 -- 172.21.15.3:0/2158355657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d58071e60 msgr2=0x7f0d5815dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 --2- 172.21.15.3:0/2158355657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d58071e60 0x7f0d5815dd70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009850 tx=0x7f0d48009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:39.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 -- 172.21.15.3:0/2158355657 shutdown_connections 2024-09-13T16:52:39.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 --2- 172.21.15.3:0/2158355657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580ce470 0x7f0d580b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 --2- 172.21.15.3:0/2158355657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d58071e60 0x7f0d5815dd70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 -- 172.21.15.3:0/2158355657 >> 172.21.15.3:0/2158355657 conn(0x7f0d580b07a0 msgr2=0x7f0d580b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:39.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 -- 172.21.15.3:0/2158355657 shutdown_connections 2024-09-13T16:52:39.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 -- 172.21.15.3:0/2158355657 wait complete. 2024-09-13T16:52:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 Processor -- start 2024-09-13T16:52:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.532+0000 7f0d5e200700 1 -- start start 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d5e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d58071e60 0x7f0d58153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d5e200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580b29d0 0x7f0d5815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d5e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d580ce470 0x7f0d5815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d5e200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d581604d0 con 0x7f0d58071e60 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d5e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d581607b0 con 0x7f0d580ce470 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d5e200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58160a90 con 0x7f0d580b29d0 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d58071e60 0x7f0d58153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d58071e60 0x7f0d58153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43798/0 (socket says 172.21.15.3:43798) 2024-09-13T16:52:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 -- 172.21.15.3:0/86936831 learned_addr learned my addr 172.21.15.3:0/86936831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57e00700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d580ce470 0x7f0d5815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d56a00700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580b29d0 0x7f0d5815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 -- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580b29d0 msgr2=0x7f0d5815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580b29d0 0x7f0d5815fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 -- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d580ce470 msgr2=0x7f0d5815bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d580ce470 0x7f0d5815bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 -- 172.21.15.3:0/86936831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480094b0 con 0x7f0d58071e60 2024-09-13T16:52:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d56a00700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580b29d0 0x7f0d5815fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57e00700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d580ce470 0x7f0d5815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d57400700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d58071e60 0x7f0d58153480 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0d48014040 tx=0x7f0d48004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d4f400700 1 -- 172.21.15.3:0/86936831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d480036c0 con 0x7f0d58071e60 2024-09-13T16:52:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d4f400700 1 -- 172.21.15.3:0/86936831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4801e070 con 0x7f0d58071e60 2024-09-13T16:52:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d58160d70 con 0x7f0d58071e60 2024-09-13T16:52:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d4f400700 1 -- 172.21.15.3:0/86936831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48003a80 con 0x7f0d58071e60 2024-09-13T16:52:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.536+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d581539c0 con 0x7f0d58071e60 2024-09-13T16:52:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.540+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5806dee0 con 0x7f0d58071e60 2024-09-13T16:52:39.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.540+0000 7f0d4f400700 1 -- 172.21.15.3:0/86936831 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d48003be0 con 0x7f0d58071e60 2024-09-13T16:52:39.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.540+0000 7f0d4f400700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0d38066730 0x7f0d38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.540+0000 7f0d4f400700 1 -- 172.21.15.3:0/86936831 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f0d48020070 con 0x7f0d58071e60 2024-09-13T16:52:39.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.544+0000 7f0d56a00700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0d38066730 0x7f0d38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.544+0000 7f0d56a00700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0d38066730 0x7f0d38068bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d581568a0 tx=0x7f0d44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.548+0000 7f0d4f400700 1 -- 172.21.15.3:0/86936831 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d480529f0 con 0x7f0d58071e60 2024-09-13T16:52:39.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.728+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d580cc9c0 con 0x7f0d58071e60 2024-09-13T16:52:39.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d4f400700 1 -- 172.21.15.3:0/86936831 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d48006ce0 con 0x7f0d58071e60 2024-09-13T16:52:39.736 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:52:39.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0d38066730 msgr2=0x7f0d38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0d38066730 0x7f0d38068bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d581568a0 tx=0x7f0d44008040 comp rx=0 tx=0).stop 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d58071e60 msgr2=0x7f0d58153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d58071e60 0x7f0d58153480 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0d48014040 tx=0x7f0d48004770 comp rx=0 tx=0).stop 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 shutdown_connections 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0d38066730 0x7f0d38068bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d580ce470 0x7f0d5815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d580b29d0 0x7f0d5815fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 --2- 172.21.15.3:0/86936831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d58071e60 0x7f0d58153480 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 >> 172.21.15.3:0/86936831 conn(0x7f0d580b07a0 msgr2=0x7f0d580ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.732+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 shutdown_connections 2024-09-13T16:52:39.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.736+0000 7f0d5e200700 1 -- 172.21.15.3:0/86936831 wait complete. 2024-09-13T16:52:39.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:52:39.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.936+0000 7fb20a400700 1 Processor -- start 2024-09-13T16:52:39.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.936+0000 7fb20a400700 1 -- start start 2024-09-13T16:52:39.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb20a400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2040b4800 0x7fb20406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb20a400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb20406d550 0x7fb20406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb20a400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2040b2db0 con 0x7fb204160fe0 2024-09-13T16:52:39.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb20a400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20406d9e0 con 0x7fb20406d550 2024-09-13T16:52:39.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb20a400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040b3050 con 0x7fb2040b4800 2024-09-13T16:52:39.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb209000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb204160fe0 0x7fb2040b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34794/0 (socket says 172.21.15.3:34794) 2024-09-13T16:52:39.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb209000700 1 -- 172.21.15.3:0/749056808 learned_addr learned my addr 172.21.15.3:0/749056808 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:39.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb209a00700 1 --2- 172.21.15.3:0/749056808 >> v2:172.21.15.3:3300/0 conn(0x7fb20406d550 0x7fb20406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb203e00700 1 --2- 172.21.15.3:0/749056808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2040b4800 0x7fb20406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb203400700 1 -- 172.21.15.3:0/749056808 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022117278 0 0) 0x7fb2040b2db0 con 0x7fb204160fe0 2024-09-13T16:52:39.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb209a00700 1 -- 172.21.15.3:0/749056808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2040b4800 msgr2=0x7fb20406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:39.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb209a00700 1 --2- 172.21.15.3:0/749056808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2040b4800 0x7fb20406ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb209a00700 1 -- 172.21.15.3:0/749056808 >> v1:172.21.15.3:6789/0 conn(0x7fb204160fe0 legacy=0x7fb2040b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.940+0000 7fb209a00700 1 -- 172.21.15.3:0/749056808 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2040b3330 con 0x7fb20406d550 2024-09-13T16:52:39.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb209a00700 1 --2- 172.21.15.3:0/749056808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb20406a2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009d90 tx=0x7fb1fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d338cacf20c258 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb203400700 1 -- 172.21.15.3:0/749056808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc00e070 con 0x7fb20406d550 2024-09-13T16:52:39.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb203400700 1 -- 172.21.15.3:0/749056808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc00bb20 con 0x7fb20406d550 2024-09-13T16:52:39.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb203400700 1 -- 172.21.15.3:0/749056808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc012430 con 0x7fb20406d550 2024-09-13T16:52:39.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 -- 172.21.15.3:0/749056808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 msgr2=0x7fb20406a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 --2- 172.21.15.3:0/749056808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb20406a2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009d90 tx=0x7fb1fc00b330 comp rx=0 tx=0).stop 2024-09-13T16:52:39.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 -- 172.21.15.3:0/749056808 shutdown_connections 2024-09-13T16:52:39.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 --2- 172.21.15.3:0/749056808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb20406a2f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 --2- 172.21.15.3:0/749056808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2040b4800 0x7fb20406ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 -- 172.21.15.3:0/749056808 >> 172.21.15.3:0/749056808 conn(0x7fb20406b400 msgr2=0x7fb20406b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:39.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 -- 172.21.15.3:0/749056808 shutdown_connections 2024-09-13T16:52:39.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 -- 172.21.15.3:0/749056808 wait complete. 2024-09-13T16:52:39.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 Processor -- start 2024-09-13T16:52:39.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.944+0000 7fb20a400700 1 -- start start 2024-09-13T16:52:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb20a400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb2040a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb20a400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb204160fe0 0x7fb2040ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb20a400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2040a9410 0x7fb2040aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb20a400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040a9970 con 0x7fb20406d550 2024-09-13T16:52:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb20a400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040ce080 con 0x7fb2040a9410 2024-09-13T16:52:39.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb20a400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040cd2c0 con 0x7fb204160fe0 2024-09-13T16:52:39.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb2040a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb2040a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43822/0 (socket says 172.21.15.3:43822) 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 -- 172.21.15.3:0/2518458511 learned_addr learned my addr 172.21.15.3:0/2518458511 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209a00700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2040a9410 0x7fb2040aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb203e00700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb204160fe0 0x7fb2040ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 -- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb204160fe0 msgr2=0x7fb2040ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb204160fe0 0x7fb2040ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 -- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2040a9410 msgr2=0x7fb2040aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2040a9410 0x7fb2040aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 -- 172.21.15.3:0/2518458511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc009a70 con 0x7fb20406d550 2024-09-13T16:52:39.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209a00700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2040a9410 0x7fb2040aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:39.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb209000700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb2040a8ed0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c4e0 tx=0x7fb1f800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb200c00700 1 -- 172.21.15.3:0/2518458511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8004ca0 con 0x7fb20406d550 2024-09-13T16:52:39.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb200c00700 1 -- 172.21.15.3:0/2518458511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f8004e20 con 0x7fb20406d550 2024-09-13T16:52:39.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2040cd5a0 con 0x7fb20406d550 2024-09-13T16:52:39.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb200c00700 1 -- 172.21.15.3:0/2518458511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8007310 con 0x7fb20406d550 2024-09-13T16:52:39.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.948+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2040cdb60 con 0x7fb20406d550 2024-09-13T16:52:39.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.952+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20406a2f0 con 0x7fb20406d550 2024-09-13T16:52:39.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.952+0000 7fb200c00700 1 -- 172.21.15.3:0/2518458511 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb1f8007470 con 0x7fb20406d550 2024-09-13T16:52:39.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.952+0000 7fb200c00700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb1e4066730 0x7fb1e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:39.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.952+0000 7fb200c00700 1 -- 172.21.15.3:0/2518458511 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb1f8084060 con 0x7fb20406d550 2024-09-13T16:52:39.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.956+0000 7fb203e00700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb1e4066730 0x7fb1e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:39.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.956+0000 7fb203e00700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb1e4066730 0x7fb1e4068bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2040cbcc0 tx=0x7fb1f0009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:39.960+0000 7fb200c00700 1 -- 172.21.15.3:0/2518458511 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb1f80b5db0 con 0x7fb20406d550 2024-09-13T16:52:40.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.140+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2040af940 con 0x7fb20406d550 2024-09-13T16:52:40.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.144+0000 7fb200c00700 1 -- 172.21.15.3:0/2518458511 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1f8051120 con 0x7fb20406d550 2024-09-13T16:52:40.149 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.144+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb1e4066730 msgr2=0x7fb1e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.144+0000 7fb20a400700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb1e4066730 0x7fb1e4068bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2040cbcc0 tx=0x7fb1f0009240 comp rx=0 tx=0).stop 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.144+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 msgr2=0x7fb2040a8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.144+0000 7fb20a400700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb2040a8ed0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c4e0 tx=0x7fb1f800c7f0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.148+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 shutdown_connections 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.148+0000 7fb20a400700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb1e4066730 0x7fb1e4068bf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.148+0000 7fb20a400700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2040a9410 0x7fb2040aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.148+0000 7fb20a400700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb204160fe0 0x7fb2040ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.148+0000 7fb20a400700 1 --2- 172.21.15.3:0/2518458511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20406d550 0x7fb2040a8ed0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.148+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 >> 172.21.15.3:0/2518458511 conn(0x7fb20406b400 msgr2=0x7fb2040b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:40.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.148+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 shutdown_connections 2024-09-13T16:52:40.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.148+0000 7fb20a400700 1 -- 172.21.15.3:0/2518458511 wait complete. 2024-09-13T16:52:40.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:52:40.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.352+0000 7fb77ee00700 1 Processor -- start 2024-09-13T16:52:40.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.352+0000 7fb77ee00700 1 -- start start 2024-09-13T16:52:40.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77ee00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb77815a6d0 0x7fb77815eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77ee00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 0x7fb77815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77ee00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77815b670 con 0x7fb77815f060 2024-09-13T16:52:40.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77ee00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780cc810 con 0x7fb77815a6d0 2024-09-13T16:52:40.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77ee00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77815b950 con 0x7fb7780cc260 2024-09-13T16:52:40.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77da00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb77815a6d0 0x7fb77815eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77d000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb77815f060 0x7fb77806d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34800/0 (socket says 172.21.15.3:34800) 2024-09-13T16:52:40.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77d000700 1 -- 172.21.15.3:0/3511047108 learned_addr learned my addr 172.21.15.3:0/3511047108 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77e400700 1 --2- 172.21.15.3:0/3511047108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 0x7fb77815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77da00700 1 -- 172.21.15.3:0/3511047108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 msgr2=0x7fb77815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77da00700 1 --2- 172.21.15.3:0/3511047108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 0x7fb77815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77da00700 1 -- 172.21.15.3:0/3511047108 >> v1:172.21.15.3:6789/0 conn(0x7fb77815f060 legacy=0x7fb77806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb777e00700 1 -- 172.21.15.3:0/3511047108 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746042433 0 0) 0x7fb77815b670 con 0x7fb77815f060 2024-09-13T16:52:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77da00700 1 -- 172.21.15.3:0/3511047108 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77815bc30 con 0x7fb77815a6d0 2024-09-13T16:52:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb77da00700 1 --2- 172.21.15.3:0/3511047108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815a6d0 0x7fb77815eb20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009850 tx=0x7fb76c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ac7046f55026d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb777e00700 1 -- 172.21.15.3:0/3511047108 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c011830 con 0x7fb77815a6d0 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb777e00700 1 -- 172.21.15.3:0/3511047108 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c011990 con 0x7fb77815a6d0 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.356+0000 7fb777e00700 1 -- 172.21.15.3:0/3511047108 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c011d20 con 0x7fb77815a6d0 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 -- 172.21.15.3:0/3511047108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815a6d0 msgr2=0x7fb77815eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 --2- 172.21.15.3:0/3511047108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815a6d0 0x7fb77815eb20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009850 tx=0x7fb76c009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 -- 172.21.15.3:0/3511047108 shutdown_connections 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 --2- 172.21.15.3:0/3511047108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 0x7fb77815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 --2- 172.21.15.3:0/3511047108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815a6d0 0x7fb77815eb20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 -- 172.21.15.3:0/3511047108 >> 172.21.15.3:0/3511047108 conn(0x7fb77806b400 msgr2=0x7fb77806b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 -- 172.21.15.3:0/3511047108 shutdown_connections 2024-09-13T16:52:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 -- 172.21.15.3:0/3511047108 wait complete. 2024-09-13T16:52:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 Processor -- start 2024-09-13T16:52:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.360+0000 7fb77ee00700 1 -- start start 2024-09-13T16:52:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77ee00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 0x7fb778145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77ee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb77815a6d0 0x7fb77815d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77ee00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815f060 0x7fb7781474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77ee00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb778145bc0 con 0x7fb77815f060 2024-09-13T16:52:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77ee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb778159820 con 0x7fb77815a6d0 2024-09-13T16:52:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77ee00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb778146440 con 0x7fb7780cc260 2024-09-13T16:52:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815f060 0x7fb7781474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815f060 0x7fb7781474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43844/0 (socket says 172.21.15.3:43844) 2024-09-13T16:52:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 -- 172.21.15.3:0/2036243552 learned_addr learned my addr 172.21.15.3:0/2036243552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77da00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 0x7fb778145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77d000700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb77815a6d0 0x7fb77815d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 -- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 msgr2=0x7fb778145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 0x7fb778145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 -- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb77815a6d0 msgr2=0x7fb77815d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb77815a6d0 0x7fb77815d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 -- 172.21.15.3:0/2036243552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0094b0 con 0x7fb77815f060 2024-09-13T16:52:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77d000700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb77815a6d0 0x7fb77815d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:40.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77e400700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815f060 0x7fb7781474e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb770007e40 tx=0x7fb77000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:40.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb775600700 1 -- 172.21.15.3:0/2036243552 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77000eea0 con 0x7fb77815f060 2024-09-13T16:52:40.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb775600700 1 -- 172.21.15.3:0/2036243552 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb770014420 con 0x7fb77815f060 2024-09-13T16:52:40.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb778146720 con 0x7fb77815f060 2024-09-13T16:52:40.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb775600700 1 -- 172.21.15.3:0/2036243552 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7700147b0 con 0x7fb77815f060 2024-09-13T16:52:40.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.364+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb778146ce0 con 0x7fb77815f060 2024-09-13T16:52:40.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.368+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77815b380 con 0x7fb77815f060 2024-09-13T16:52:40.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.368+0000 7fb775600700 1 -- 172.21.15.3:0/2036243552 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb770014910 con 0x7fb77815f060 2024-09-13T16:52:40.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.372+0000 7fb775600700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb758066730 0x7fb758068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.372+0000 7fb775600700 1 -- 172.21.15.3:0/2036243552 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb770084c60 con 0x7fb77815f060 2024-09-13T16:52:40.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.372+0000 7fb77da00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb758066730 0x7fb758068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.372+0000 7fb77da00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb758066730 0x7fb758068bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb77815dc00 tx=0x7fb76c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:40.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.380+0000 7fb775600700 1 -- 172.21.15.3:0/2036243552 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb770051d20 con 0x7fb77815f060 2024-09-13T16:52:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.564+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb778071e90 con 0x7fb77815f060 2024-09-13T16:52:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb775600700 1 -- 172.21.15.3:0/2036243552 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb770055340 con 0x7fb77815f060 2024-09-13T16:52:40.572 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:52:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb758066730 msgr2=0x7fb758068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb758066730 0x7fb758068bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb77815dc00 tx=0x7fb76c008030 comp rx=0 tx=0).stop 2024-09-13T16:52:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815f060 msgr2=0x7fb7781474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815f060 0x7fb7781474e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb770007e40 tx=0x7fb77000e3c0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 shutdown_connections 2024-09-13T16:52:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb758066730 0x7fb758068bf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77815f060 0x7fb7781474e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb77815a6d0 0x7fb77815d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 --2- 172.21.15.3:0/2036243552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7780cc260 0x7fb778145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 >> 172.21.15.3:0/2036243552 conn(0x7fb77806b400 msgr2=0x7fb7780ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 shutdown_connections 2024-09-13T16:52:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.568+0000 7fb77ee00700 1 -- 172.21.15.3:0/2036243552 wait complete. 2024-09-13T16:52:40.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:52:40.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21fe00700 1 Processor -- start 2024-09-13T16:52:40.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21fe00700 1 -- start start 2024-09-13T16:52:40.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe22006ba80 0x7fe22006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21fe00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2200ce5d0 con 0x7fe220070dc0 2024-09-13T16:52:40.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21fe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22015af00 con 0x7fe22006be60 2024-09-13T16:52:40.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21fe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2200cd9c0 con 0x7fe22006ba80 2024-09-13T16:52:40.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21f400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe220070dc0 0x7fe2201646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34808/0 (socket says 172.21.15.3:34808) 2024-09-13T16:52:40.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21f400700 1 -- 172.21.15.3:0/38884500 learned_addr learned my addr 172.21.15.3:0/38884500 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:40.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21ea00700 1 --2- 172.21.15.3:0/38884500 >> v2:172.21.15.157:3300/0 conn(0x7fe22006ba80 0x7fe22006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21d600700 1 -- 172.21.15.3:0/38884500 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869357309 0 0) 0x7fe2200ce5d0 con 0x7fe220070dc0 2024-09-13T16:52:40.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21d600700 1 -- 172.21.15.3:0/38884500 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2040035d0 con 0x7fe220070dc0 2024-09-13T16:52:40.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.776+0000 7fe21d600700 1 -- 172.21.15.3:0/38884500 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144965975 0 0) 0x7fe22015af00 con 0x7fe22006be60 2024-09-13T16:52:40.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21d600700 1 -- 172.21.15.3:0/38884500 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2200ce5d0 con 0x7fe22006be60 2024-09-13T16:52:40.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21ea00700 1 -- 172.21.15.3:0/38884500 >> v1:172.21.15.157:6789/0 conn(0x7fe22006be60 legacy=0x7fe22006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21ea00700 1 -- 172.21.15.3:0/38884500 >> v1:172.21.15.3:6789/0 conn(0x7fe220070dc0 legacy=0x7fe2201646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21ea00700 1 -- 172.21.15.3:0/38884500 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200cdca0 con 0x7fe22006ba80 2024-09-13T16:52:40.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21ea00700 1 --2- 172.21.15.3:0/38884500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22006ba80 0x7fe22006d6b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe210009850 tx=0x7fe210009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e06764d5c693189 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:40.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21d600700 1 -- 172.21.15.3:0/38884500 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210011830 con 0x7fe22006ba80 2024-09-13T16:52:40.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21d600700 1 -- 172.21.15.3:0/38884500 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe210011990 con 0x7fe22006ba80 2024-09-13T16:52:40.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21d600700 1 -- 172.21.15.3:0/38884500 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210011d40 con 0x7fe22006ba80 2024-09-13T16:52:40.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21fe00700 1 -- 172.21.15.3:0/38884500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22006ba80 msgr2=0x7fe22006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21fe00700 1 --2- 172.21.15.3:0/38884500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22006ba80 0x7fe22006d6b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe210009850 tx=0x7fe210009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:40.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21fe00700 1 -- 172.21.15.3:0/38884500 shutdown_connections 2024-09-13T16:52:40.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21fe00700 1 --2- 172.21.15.3:0/38884500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22006ba80 0x7fe22006d6b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21fe00700 1 -- 172.21.15.3:0/38884500 >> 172.21.15.3:0/38884500 conn(0x7fe22006a2f0 msgr2=0x7fe22006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:40.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21fe00700 1 -- 172.21.15.3:0/38884500 shutdown_connections 2024-09-13T16:52:40.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.780+0000 7fe21fe00700 1 -- 172.21.15.3:0/38884500 wait complete. 2024-09-13T16:52:40.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21fe00700 1 Processor -- start 2024-09-13T16:52:40.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21fe00700 1 -- start start 2024-09-13T16:52:40.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22006ba80 0x7fe22014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22006be60 0x7fe220153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220070dc0 0x7fe220158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21fe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22014ded0 con 0x7fe22006ba80 2024-09-13T16:52:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2201543b0 con 0x7fe220070dc0 2024-09-13T16:52:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21fe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe220154690 con 0x7fe22006be60 2024-09-13T16:52:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22006ba80 0x7fe22014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22006ba80 0x7fe22014d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43858/0 (socket says 172.21.15.3:43858) 2024-09-13T16:52:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21ea00700 1 -- 172.21.15.3:0/2851908938 learned_addr learned my addr 172.21.15.3:0/2851908938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21f400700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220070dc0 0x7fe220158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.784+0000 7fe21e000700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22006be60 0x7fe220153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21ea00700 1 -- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22006be60 msgr2=0x7fe220153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21ea00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22006be60 0x7fe220153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21ea00700 1 -- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220070dc0 msgr2=0x7fe220158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21ea00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220070dc0 0x7fe220158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21ea00700 1 -- 172.21.15.3:0/2851908938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100094b0 con 0x7fe22006ba80 2024-09-13T16:52:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21e000700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22006be60 0x7fe220153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21ea00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22006ba80 0x7fe22014d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe220071250 tx=0x7fe210004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe216a00700 1 -- 172.21.15.3:0/2851908938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21001aac0 con 0x7fe22006ba80 2024-09-13T16:52:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe216a00700 1 -- 172.21.15.3:0/2851908938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21001f070 con 0x7fe22006ba80 2024-09-13T16:52:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe220154970 con 0x7fe22006ba80 2024-09-13T16:52:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe216a00700 1 -- 172.21.15.3:0/2851908938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21001ae60 con 0x7fe22006ba80 2024-09-13T16:52:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe220154f30 con 0x7fe22006ba80 2024-09-13T16:52:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.788+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe220155150 con 0x7fe22006ba80 2024-09-13T16:52:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.792+0000 7fe216a00700 1 -- 172.21.15.3:0/2851908938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe21001b430 con 0x7fe22006ba80 2024-09-13T16:52:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.792+0000 7fe216a00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe200066730 0x7fe200068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.792+0000 7fe216a00700 1 -- 172.21.15.3:0/2851908938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fe210085440 con 0x7fe22006ba80 2024-09-13T16:52:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.792+0000 7fe21e000700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe200066730 0x7fe200068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:40.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.796+0000 7fe21e000700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe200066730 0x7fe200068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2200adb00 tx=0x7fe20c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:40.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.796+0000 7fe216a00700 1 -- 172.21.15.3:0/2851908938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe210052590 con 0x7fe22006ba80 2024-09-13T16:52:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.996+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe22006c2a0 con 0x7fe22006ba80 2024-09-13T16:52:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:40.996+0000 7fe216a00700 1 -- 172.21.15.3:0/2851908938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v46) v1 ==== 80+0+118 (secure 0 0 0) 0x7fe210055bb0 con 0x7fe22006ba80 2024-09-13T16:52:41.001 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6835/750657669 2024-09-14T16:52:11.310659+0000 2024-09-13T16:52:41.001 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6834/750657669 2024-09-14T16:52:11.310659+0000 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe200066730 msgr2=0x7fe200068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe200066730 0x7fe200068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2200adb00 tx=0x7fe20c009040 comp rx=0 tx=0).stop 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22006ba80 msgr2=0x7fe22014d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22006ba80 0x7fe22014d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe220071250 tx=0x7fe210004770 comp rx=0 tx=0).stop 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 shutdown_connections 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe200066730 0x7fe200068bf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220070dc0 0x7fe220158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22006be60 0x7fe220153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 --2- 172.21.15.3:0/2851908938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22006ba80 0x7fe22014d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 >> 172.21.15.3:0/2851908938 conn(0x7fe22006a2f0 msgr2=0x7fe2200705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:41.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 shutdown_connections 2024-09-13T16:52:41.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.000+0000 7fe21fe00700 1 -- 172.21.15.3:0/2851908938 wait complete. 2024-09-13T16:52:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:listed 2 entries 2024-09-13T16:52:41.030 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T16:52:41.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:41.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T16:52:41.109 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:52:41.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:52:41.110 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T16:52:41.127 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:52:41.127 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:41.141 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346844k,nr_inodes=4086711,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T16:52:41.141 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:41.141 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:41.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349932k,nr_inodes=4087483,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4619 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T16:52:41.192 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:52:41.192 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:41.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346844k,nr_inodes=4086711,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T16:52:41.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.452+0000 7f871dc00700 1 Processor -- start 2024-09-13T16:52:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.452+0000 7f871dc00700 1 -- start start 2024-09-13T16:52:41.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871dc00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8718165770 0x7f8718164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871dc00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8718165110 0x7f8718070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871dc00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871806ba80 con 0x7f8718160f30 2024-09-13T16:52:41.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871dc00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871806fad0 con 0x7f8718165110 2024-09-13T16:52:41.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871dc00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8718165550 con 0x7f8718165770 2024-09-13T16:52:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871d200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8718165110 0x7f8718070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871d200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8718165110 0x7f8718070240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43872/0 (socket says 172.21.15.3:43872) 2024-09-13T16:52:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871d200700 1 -- 172.21.15.3:0/1170767745 learned_addr learned my addr 172.21.15.3:0/1170767745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f8717e00700 1 --1- 172.21.15.3:0/1170767745 >> v1:172.21.15.157:6789/0 conn(0x7f8718160f30 0x7f871806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:52312/0 (socket says 172.21.15.3:52312) 2024-09-13T16:52:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f870f400700 1 --2- 172.21.15.3:0/1170767745 >> v2:172.21.15.157:3300/0 conn(0x7f8718165770 0x7f8718164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871d200700 1 -- 172.21.15.3:0/1170767745 >> v2:172.21.15.157:3300/0 conn(0x7f8718165770 msgr2=0x7f8718164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871d200700 1 --2- 172.21.15.3:0/1170767745 >> v2:172.21.15.157:3300/0 conn(0x7f8718165770 0x7f8718164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871d200700 1 -- 172.21.15.3:0/1170767745 >> v1:172.21.15.157:6789/0 conn(0x7f8718160f30 legacy=0x7f871806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871d200700 1 -- 172.21.15.3:0/1170767745 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871806bcd0 con 0x7f8718165110 2024-09-13T16:52:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.456+0000 7f871d200700 1 --2- 172.21.15.3:0/1170767745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165110 0x7f8718070240 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87180b3810 tx=0x7f8710009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dead3b0303490cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f8717400700 1 -- 172.21.15.3:0/1170767745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710011830 con 0x7f8718165110 2024-09-13T16:52:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f8717400700 1 -- 172.21.15.3:0/1170767745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8710011990 con 0x7f8718165110 2024-09-13T16:52:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f8717400700 1 -- 172.21.15.3:0/1170767745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710011d20 con 0x7f8718165110 2024-09-13T16:52:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f871dc00700 1 -- 172.21.15.3:0/1170767745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165110 msgr2=0x7f8718070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f871dc00700 1 --2- 172.21.15.3:0/1170767745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165110 0x7f8718070240 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87180b3810 tx=0x7f8710009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f871dc00700 1 -- 172.21.15.3:0/1170767745 shutdown_connections 2024-09-13T16:52:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f871dc00700 1 --2- 172.21.15.3:0/1170767745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165110 0x7f8718070240 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f871dc00700 1 --2- 172.21.15.3:0/1170767745 >> v2:172.21.15.157:3300/0 conn(0x7f8718165770 0x7f8718164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f871dc00700 1 -- 172.21.15.3:0/1170767745 >> 172.21.15.3:0/1170767745 conn(0x7f871806a2f0 msgr2=0x7f871806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f871dc00700 1 -- 172.21.15.3:0/1170767745 shutdown_connections 2024-09-13T16:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.460+0000 7f871dc00700 1 -- 172.21.15.3:0/1170767745 wait complete. 2024-09-13T16:52:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.464+0000 7f871dc00700 1 Processor -- start 2024-09-13T16:52:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871dc00700 1 -- start start 2024-09-13T16:52:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871dc00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8718160f30 0x7f871814bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8718165110 0x7f8718158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871dc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165770 0x7f87181eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871dc00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87181583e0 con 0x7f8718165770 2024-09-13T16:52:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871dc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871815a8b0 con 0x7f8718165110 2024-09-13T16:52:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871dc00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871815a0a0 con 0x7f8718160f30 2024-09-13T16:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f8717e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8718160f30 0x7f871814bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871d200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165770 0x7f87181eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f8717e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8718160f30 0x7f871814bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:50490/0 (socket says 172.21.15.3:50490) 2024-09-13T16:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f870f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8718165110 0x7f8718158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f8717e00700 1 -- 172.21.15.3:0/3002720929 learned_addr learned my addr 172.21.15.3:0/3002720929 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871d200700 1 -- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8718160f30 msgr2=0x7f871814bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871d200700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8718160f30 0x7f871814bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871d200700 1 -- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8718165110 msgr2=0x7f8718158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871d200700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8718165110 0x7f8718158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871d200700 1 -- 172.21.15.3:0/3002720929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710009570 con 0x7f8718165770 2024-09-13T16:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.468+0000 7f871d200700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165770 0x7f87181eb9b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8710016000 tx=0x7f8710004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.472+0000 7f8714c00700 1 -- 172.21.15.3:0/3002720929 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710004450 con 0x7f8718165770 2024-09-13T16:52:41.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.472+0000 7f8714c00700 1 -- 172.21.15.3:0/3002720929 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87100045b0 con 0x7f8718165770 2024-09-13T16:52:41.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.472+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8718158940 con 0x7f8718165770 2024-09-13T16:52:41.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.472+0000 7f8714c00700 1 -- 172.21.15.3:0/3002720929 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710003900 con 0x7f8718165770 2024-09-13T16:52:41.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.472+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871815a2f0 con 0x7f8718165770 2024-09-13T16:52:41.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.472+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8718158ad0 con 0x7f8718165770 2024-09-13T16:52:41.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.476+0000 7f8714c00700 1 -- 172.21.15.3:0/3002720929 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8710021070 con 0x7f8718165770 2024-09-13T16:52:41.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.476+0000 7f8714c00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86f4066730 0x7f86f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.476+0000 7f8714c00700 1 -- 172.21.15.3:0/3002720929 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f8710085880 con 0x7f8718165770 2024-09-13T16:52:41.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.476+0000 7f8717e00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86f4066730 0x7f86f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.476+0000 7f8717e00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86f4066730 0x7f86f4068bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f87080012e0 tx=0x7f8708001be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.484+0000 7f8714c00700 1 -- 172.21.15.3:0/3002720929 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f87100529d0 con 0x7f8718165770 2024-09-13T16:52:41.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.720+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8718164bd0 con 0x7f8718165770 2024-09-13T16:52:41.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.720+0000 7f8714c00700 1 -- 172.21.15.3:0/3002720929 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f8710055ff0 con 0x7f8718165770 2024-09-13T16:52:41.727 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:41.727 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:15.362813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5569":{"gid":5569,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/1311223400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1311223400},{"type":"v1","addr":"172.21.15.3:6835","nonce":1311223400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86f4066730 msgr2=0x7f86f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86f4066730 0x7f86f4068bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f87080012e0 tx=0x7f8708001be0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165770 msgr2=0x7f87181eb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165770 0x7f87181eb9b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8710016000 tx=0x7f8710004770 comp rx=0 tx=0).stop 2024-09-13T16:52:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 shutdown_connections 2024-09-13T16:52:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86f4066730 0x7f86f4068bf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8718165770 0x7f87181eb9b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8718165110 0x7f8718158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 --2- 172.21.15.3:0/3002720929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8718160f30 0x7f871814bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 >> 172.21.15.3:0/3002720929 conn(0x7f871806a2f0 msgr2=0x7f87180b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 shutdown_connections 2024-09-13T16:52:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.724+0000 7f871dc00700 1 -- 172.21.15.3:0/3002720929 wait complete. 2024-09-13T16:52:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-09-13T16:52:41.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:41.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.924+0000 7fcfbce00700 1 Processor -- start 2024-09-13T16:52:41.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfbce00700 1 -- start start 2024-09-13T16:52:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfbce00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfb815cc40 0x7fcfb806fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfbce00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 0x7fcfb80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfbce00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb80adc60 con 0x7fcfb815d020 2024-09-13T16:52:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfbce00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb806da50 con 0x7fcfb815cc40 2024-09-13T16:52:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfbce00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb806dd30 con 0x7fcfb806d540 2024-09-13T16:52:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfb7400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfb815cc40 0x7fcfb806fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfb7400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfb815cc40 0x7fcfb806fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43890/0 (socket says 172.21.15.3:43890) 2024-09-13T16:52:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfb7400700 1 -- 172.21.15.3:0/1629325180 learned_addr learned my addr 172.21.15.3:0/1629325180 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfaea00700 1 --2- 172.21.15.3:0/1629325180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 0x7fcfb80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfb7400700 1 -- 172.21.15.3:0/1629325180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 msgr2=0x7fcfb80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfb7400700 1 --2- 172.21.15.3:0/1629325180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 0x7fcfb80ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfb7400700 1 -- 172.21.15.3:0/1629325180 >> v1:172.21.15.157:6789/0 conn(0x7fcfb815d020 legacy=0x7fcfb81609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfb7400700 1 -- 172.21.15.3:0/1629325180 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80ad610 con 0x7fcfb815cc40 2024-09-13T16:52:41.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.928+0000 7fcfb7400700 1 --2- 172.21.15.3:0/1629325180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815cc40 0x7fcfb806fc20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009e50 tx=0x7fcfa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac34eb2f5ba80810 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfb6a00700 1 -- 172.21.15.3:0/1629325180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800e070 con 0x7fcfb815cc40 2024-09-13T16:52:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfb6a00700 1 -- 172.21.15.3:0/1629325180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa800b910 con 0x7fcfb815cc40 2024-09-13T16:52:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfb6a00700 1 -- 172.21.15.3:0/1629325180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800bcc0 con 0x7fcfb815cc40 2024-09-13T16:52:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfbce00700 1 -- 172.21.15.3:0/1629325180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815cc40 msgr2=0x7fcfb806fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1629325180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815cc40 0x7fcfb806fc20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009e50 tx=0x7fcfa800b330 comp rx=0 tx=0).stop 2024-09-13T16:52:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfbce00700 1 -- 172.21.15.3:0/1629325180 shutdown_connections 2024-09-13T16:52:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1629325180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 0x7fcfb80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1629325180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815cc40 0x7fcfb806fc20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfbce00700 1 -- 172.21.15.3:0/1629325180 >> 172.21.15.3:0/1629325180 conn(0x7fcfb806a2f0 msgr2=0x7fcfb806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfbce00700 1 -- 172.21.15.3:0/1629325180 shutdown_connections 2024-09-13T16:52:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.932+0000 7fcfbce00700 1 -- 172.21.15.3:0/1629325180 wait complete. 2024-09-13T16:52:41.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfbce00700 1 Processor -- start 2024-09-13T16:52:41.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfbce00700 1 -- start start 2024-09-13T16:52:41.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfbce00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 0x7fcfb81f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfbce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815cc40 0x7fcfb815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfbce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815d020 0x7fcfb80ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfbce00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb815e9c0 con 0x7fcfb815d020 2024-09-13T16:52:41.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfbce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81f6800 con 0x7fcfb815cc40 2024-09-13T16:52:41.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfbce00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81f6ae0 con 0x7fcfb806d540 2024-09-13T16:52:41.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfb7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815d020 0x7fcfb80ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfb7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815d020 0x7fcfb80ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43892/0 (socket says 172.21.15.3:43892) 2024-09-13T16:52:41.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.936+0000 7fcfb7e00700 1 -- 172.21.15.3:0/1656807948 learned_addr learned my addr 172.21.15.3:0/1656807948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:41.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfaea00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815cc40 0x7fcfb815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfb7400700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 0x7fcfb81f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfb7e00700 1 -- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 msgr2=0x7fcfb81f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfb7e00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 0x7fcfb81f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfb7e00700 1 -- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815cc40 msgr2=0x7fcfb815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:41.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfb7e00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815cc40 0x7fcfb815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:41.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfb7e00700 1 -- 172.21.15.3:0/1656807948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8009b30 con 0x7fcfb815d020 2024-09-13T16:52:41.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfb7e00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815d020 0x7fcfb80ae8a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c390 tx=0x7fcfb000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:41.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfafe00700 1 -- 172.21.15.3:0/1656807948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00052d0 con 0x7fcfb815d020 2024-09-13T16:52:41.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfafe00700 1 -- 172.21.15.3:0/1656807948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb0005430 con 0x7fcfb815d020 2024-09-13T16:52:41.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfafe00700 1 -- 172.21.15.3:0/1656807948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00055a0 con 0x7fcfb815d020 2024-09-13T16:52:41.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb815ec30 con 0x7fcfb815d020 2024-09-13T16:52:41.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb81f6dc0 con 0x7fcfb815d020 2024-09-13T16:52:41.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.940+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb815ee50 con 0x7fcfb815d020 2024-09-13T16:52:41.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.944+0000 7fcfafe00700 1 -- 172.21.15.3:0/1656807948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcfb00038c0 con 0x7fcfb815d020 2024-09-13T16:52:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.944+0000 7fcfafe00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf94066760 0x7fcf94068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.948+0000 7fcfafe00700 1 -- 172.21.15.3:0/1656807948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fcfb00834f0 con 0x7fcfb815d020 2024-09-13T16:52:41.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.948+0000 7fcfb7400700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf94066760 0x7fcf94068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:41.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.948+0000 7fcfb7400700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf94066760 0x7fcf94068c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800b370 tx=0x7fcfa8009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:41.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:41.952+0000 7fcfafe00700 1 -- 172.21.15.3:0/1656807948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcfb00506c0 con 0x7fcfb815d020 2024-09-13T16:52:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.144+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfb806fc20 con 0x7fcfb815d020 2024-09-13T16:52:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.148+0000 7fcfafe00700 1 -- 172.21.15.3:0/1656807948 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16900 (secure 0 0 0) 0x7fcfb0053ce0 con 0x7fcfb815d020 2024-09-13T16:52:42.155 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:42.155 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":46,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:15.362510+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:01.009263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:02.059017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/750657669":"2024-09-14T16:52:11.310659+0000","172.21.15.3:6834/750657669":"2024-09-14T16:52:11.310659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf94066760 msgr2=0x7fcf94068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf94066760 0x7fcf94068c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800b370 tx=0x7fcfa8009000 comp rx=0 tx=0).stop 2024-09-13T16:52:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815d020 msgr2=0x7fcfb80ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815d020 0x7fcfb80ae8a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c390 tx=0x7fcfb000c6a0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 shutdown_connections 2024-09-13T16:52:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf94066760 0x7fcf94068c20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfb815d020 0x7fcfb80ae8a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815cc40 0x7fcfb815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 --2- 172.21.15.3:0/1656807948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb806d540 0x7fcfb81f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 >> 172.21.15.3:0/1656807948 conn(0x7fcfb806a2f0 msgr2=0x7fcfb8160910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 shutdown_connections 2024-09-13T16:52:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.152+0000 7fcfbce00700 1 -- 172.21.15.3:0/1656807948 wait complete. 2024-09-13T16:52:42.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:52:01.009263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:52:02.059017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:52:42.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:52:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.356+0000 7f898da00700 1 Processor -- start 2024-09-13T16:52:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.356+0000 7f898da00700 1 -- start start 2024-09-13T16:52:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.356+0000 7f898da00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8988070dc0 0x7f898806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.356+0000 7f898da00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f898806c3a0 0x7f89881646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.356+0000 7f898da00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89880ce5d0 con 0x7f898806ba80 2024-09-13T16:52:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.356+0000 7f898da00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898806ddc0 con 0x7f8988070dc0 2024-09-13T16:52:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.356+0000 7f898da00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898815af00 con 0x7f898806c3a0 2024-09-13T16:52:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.356+0000 7f8987400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8988070dc0 0x7f898806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8987400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8988070dc0 0x7f898806be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43894/0 (socket says 172.21.15.3:43894) 2024-09-13T16:52:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8987400700 1 -- 172.21.15.3:0/3744461195 learned_addr learned my addr 172.21.15.3:0/3744461195 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8987e00700 1 --1- 172.21.15.3:0/3744461195 >> v1:172.21.15.3:6789/0 conn(0x7f898806ba80 0x7f898806d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34816/0 (socket says 172.21.15.3:34816) 2024-09-13T16:52:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898d000700 1 --2- 172.21.15.3:0/3744461195 >> v2:172.21.15.157:3300/0 conn(0x7f898806c3a0 0x7f89881646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8987400700 1 -- 172.21.15.3:0/3744461195 >> v2:172.21.15.157:3300/0 conn(0x7f898806c3a0 msgr2=0x7f89881646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8987400700 1 --2- 172.21.15.3:0/3744461195 >> v2:172.21.15.157:3300/0 conn(0x7f898806c3a0 0x7f89881646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8987400700 1 -- 172.21.15.3:0/3744461195 >> v1:172.21.15.3:6789/0 conn(0x7f898806ba80 legacy=0x7f898806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8987400700 1 -- 172.21.15.3:0/3744461195 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89880cd9c0 con 0x7f8988070dc0 2024-09-13T16:52:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8986a00700 1 -- 172.21.15.3:0/3744461195 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39208937 0 0) 0x7f89880ce5d0 con 0x7f898806ba80 2024-09-13T16:52:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8987400700 1 --2- 172.21.15.3:0/3744461195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8988070dc0 0x7f898806be60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8988072210 tx=0x7f8974009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad37f515b4ce299 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8986a00700 1 -- 172.21.15.3:0/3744461195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974011830 con 0x7f8988070dc0 2024-09-13T16:52:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8986a00700 1 -- 172.21.15.3:0/3744461195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8974011990 con 0x7f8988070dc0 2024-09-13T16:52:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f8986a00700 1 -- 172.21.15.3:0/3744461195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974011d20 con 0x7f8988070dc0 2024-09-13T16:52:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898da00700 1 -- 172.21.15.3:0/3744461195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8988070dc0 msgr2=0x7f898806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898da00700 1 --2- 172.21.15.3:0/3744461195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8988070dc0 0x7f898806be60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8988072210 tx=0x7f8974009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898da00700 1 -- 172.21.15.3:0/3744461195 shutdown_connections 2024-09-13T16:52:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898da00700 1 --2- 172.21.15.3:0/3744461195 >> v2:172.21.15.157:3300/0 conn(0x7f898806c3a0 0x7f89881646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898da00700 1 --2- 172.21.15.3:0/3744461195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8988070dc0 0x7f898806be60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898da00700 1 -- 172.21.15.3:0/3744461195 >> 172.21.15.3:0/3744461195 conn(0x7f898806a2f0 msgr2=0x7f898806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898da00700 1 -- 172.21.15.3:0/3744461195 shutdown_connections 2024-09-13T16:52:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.360+0000 7f898da00700 1 -- 172.21.15.3:0/3744461195 wait complete. 2024-09-13T16:52:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898da00700 1 Processor -- start 2024-09-13T16:52:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898da00700 1 -- start start 2024-09-13T16:52:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898da00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898806ba80 0x7f89880b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898da00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898806c3a0 0x7f898814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898da00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8988070dc0 0x7f898814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898da00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881379e0 con 0x7f898806c3a0 2024-09-13T16:52:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898da00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89880b1df0 con 0x7f8988070dc0 2024-09-13T16:52:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898da00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89880b20d0 con 0x7f898806ba80 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898806c3a0 0x7f898814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898806c3a0 0x7f898814ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43896/0 (socket says 172.21.15.3:43896) 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987400700 1 -- 172.21.15.3:0/822734492 learned_addr learned my addr 172.21.15.3:0/822734492 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987e00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898806ba80 0x7f89880b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f898d000700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8988070dc0 0x7f898814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987400700 1 -- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898806ba80 msgr2=0x7f89880b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987400700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898806ba80 0x7f89880b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987400700 1 -- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8988070dc0 msgr2=0x7f898814bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987400700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8988070dc0 0x7f898814bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.364+0000 7f8987400700 1 -- 172.21.15.3:0/822734492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8974009570 con 0x7f898806c3a0 2024-09-13T16:52:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.368+0000 7f8987400700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898806c3a0 0x7f898814ba00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8974000c00 tx=0x7f8974004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.368+0000 7f897fe00700 1 -- 172.21.15.3:0/822734492 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974004610 con 0x7f898806c3a0 2024-09-13T16:52:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.368+0000 7f897fe00700 1 -- 172.21.15.3:0/822734492 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897401e070 con 0x7f898806c3a0 2024-09-13T16:52:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.368+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89880b23b0 con 0x7f898806c3a0 2024-09-13T16:52:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.368+0000 7f897fe00700 1 -- 172.21.15.3:0/822734492 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89740038c0 con 0x7f898806c3a0 2024-09-13T16:52:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.368+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89880b2970 con 0x7f898806c3a0 2024-09-13T16:52:42.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.368+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89880cd5e0 con 0x7f898806c3a0 2024-09-13T16:52:42.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.372+0000 7f897fe00700 1 -- 172.21.15.3:0/822734492 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8974003a20 con 0x7f898806c3a0 2024-09-13T16:52:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.372+0000 7f897fe00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8968066760 0x7f8968068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.372+0000 7f897fe00700 1 -- 172.21.15.3:0/822734492 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f8974020070 con 0x7f898806c3a0 2024-09-13T16:52:42.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.372+0000 7f8987e00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8968066760 0x7f8968068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.372+0000 7f8987e00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8968066760 0x7f8968068c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8978000f60 tx=0x7f8978007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:42.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.376+0000 7f897fe00700 1 -- 172.21.15.3:0/822734492 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8974052880 con 0x7f898806c3a0 2024-09-13T16:52:42.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.576+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89880ca8a0 con 0x7f898806c3a0 2024-09-13T16:52:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.576+0000 7f897fe00700 1 -- 172.21.15.3:0/822734492 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8974055ea0 con 0x7f898806c3a0 2024-09-13T16:52:42.581 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:52:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.580+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8968066760 msgr2=0x7f8968068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.580+0000 7f898da00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8968066760 0x7f8968068c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8978000f60 tx=0x7f8978007c00 comp rx=0 tx=0).stop 2024-09-13T16:52:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.580+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898806c3a0 msgr2=0x7f898814ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.580+0000 7f898da00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898806c3a0 0x7f898814ba00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8974000c00 tx=0x7f8974004770 comp rx=0 tx=0).stop 2024-09-13T16:52:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.584+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 shutdown_connections 2024-09-13T16:52:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.584+0000 7f898da00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8968066760 0x7f8968068c20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.584+0000 7f898da00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8988070dc0 0x7f898814bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.584+0000 7f898da00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898806c3a0 0x7f898814ba00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.584+0000 7f898da00700 1 --2- 172.21.15.3:0/822734492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f898806ba80 0x7f89880b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.584+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 >> 172.21.15.3:0/822734492 conn(0x7f898806a2f0 msgr2=0x7f898806a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.584+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 shutdown_connections 2024-09-13T16:52:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.584+0000 7f898da00700 1 -- 172.21.15.3:0/822734492 wait complete. 2024-09-13T16:52:42.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:52:42.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.788+0000 7fb51b200700 1 Processor -- start 2024-09-13T16:52:42.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.788+0000 7fb51b200700 1 -- start start 2024-09-13T16:52:42.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51b200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c15aca0 0x7fb51c0cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51b200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb51c06eed0 0x7fb51c0cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51b200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c06dd60 con 0x7fb51c06ea90 2024-09-13T16:52:42.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51b200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0cc5a0 con 0x7fb51c06eed0 2024-09-13T16:52:42.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51b200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0b3250 con 0x7fb51c15aca0 2024-09-13T16:52:42.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51a800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb51c06eed0 0x7fb51c0cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51a800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb51c06eed0 0x7fb51c0cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43902/0 (socket says 172.21.15.3:43902) 2024-09-13T16:52:42.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51a800700 1 -- 172.21.15.3:0/1697916836 learned_addr learned my addr 172.21.15.3:0/1697916836 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:42.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb519e00700 1 --2- 172.21.15.3:0/1697916836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c15aca0 0x7fb51c0cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51a800700 1 -- 172.21.15.3:0/1697916836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c15aca0 msgr2=0x7fb51c0cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:42.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb518a00700 1 -- 172.21.15.3:0/1697916836 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329263902 0 0) 0x7fb51c06dd60 con 0x7fb51c06ea90 2024-09-13T16:52:42.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51a800700 1 --2- 172.21.15.3:0/1697916836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c15aca0 0x7fb51c0cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51a800700 1 -- 172.21.15.3:0/1697916836 >> v1:172.21.15.157:6789/0 conn(0x7fb51c06ea90 legacy=0x7fb51c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51a800700 1 -- 172.21.15.3:0/1697916836 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0cc7b0 con 0x7fb51c06eed0 2024-09-13T16:52:42.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb51a800700 1 --2- 172.21.15.3:0/1697916836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0cefc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb514009e50 tx=0x7fb51400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a7aa1b7a85a173 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:42.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb518a00700 1 -- 172.21.15.3:0/1697916836 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400e070 con 0x7fb51c06eed0 2024-09-13T16:52:42.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb518a00700 1 -- 172.21.15.3:0/1697916836 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51400b910 con 0x7fb51c06eed0 2024-09-13T16:52:42.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.792+0000 7fb518a00700 1 -- 172.21.15.3:0/1697916836 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400bcc0 con 0x7fb51c06eed0 2024-09-13T16:52:42.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 -- 172.21.15.3:0/1697916836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 msgr2=0x7fb51c0cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 --2- 172.21.15.3:0/1697916836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0cefc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb514009e50 tx=0x7fb51400b330 comp rx=0 tx=0).stop 2024-09-13T16:52:42.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 -- 172.21.15.3:0/1697916836 shutdown_connections 2024-09-13T16:52:42.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 --2- 172.21.15.3:0/1697916836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0cefc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 --2- 172.21.15.3:0/1697916836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c15aca0 0x7fb51c0cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 -- 172.21.15.3:0/1697916836 >> 172.21.15.3:0/1697916836 conn(0x7fb51c0b24c0 msgr2=0x7fb51c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:42.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 -- 172.21.15.3:0/1697916836 shutdown_connections 2024-09-13T16:52:42.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 -- 172.21.15.3:0/1697916836 wait complete. 2024-09-13T16:52:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 Processor -- start 2024-09-13T16:52:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.796+0000 7fb51b200700 1 -- start start 2024-09-13T16:52:42.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51b200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c06ea90 0x7fb51c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51b200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb51c15aca0 0x7fb51c0ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51b200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0c6220 con 0x7fb51c06eed0 2024-09-13T16:52:42.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c06c380 con 0x7fb51c15aca0 2024-09-13T16:52:42.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51b200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c158470 con 0x7fb51c06ea90 2024-09-13T16:52:42.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43912/0 (socket says 172.21.15.3:43912) 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 -- 172.21.15.3:0/2942263427 learned_addr learned my addr 172.21.15.3:0/2942263427 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519e00700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c06ea90 0x7fb51c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51a800700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb51c15aca0 0x7fb51c0ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 -- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c06ea90 msgr2=0x7fb51c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c06ea90 0x7fb51c157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 -- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb51c15aca0 msgr2=0x7fb51c0ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb51c15aca0 0x7fb51c0ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 -- 172.21.15.3:0/2942263427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514009b30 con 0x7fb51c06eed0 2024-09-13T16:52:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51a800700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb51c15aca0 0x7fb51c0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb519400700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0c6640 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c4a0 tx=0x7fb50800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb512000700 1 -- 172.21.15.3:0/2942263427 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5080054e0 con 0x7fb51c06eed0 2024-09-13T16:52:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb512000700 1 -- 172.21.15.3:0/2942263427 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5080036a0 con 0x7fb51c06eed0 2024-09-13T16:52:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb512000700 1 -- 172.21.15.3:0/2942263427 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508003a50 con 0x7fb51c06eed0 2024-09-13T16:52:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c0c55a0 con 0x7fb51c06eed0 2024-09-13T16:52:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.800+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c0c5b60 con 0x7fb51c06eed0 2024-09-13T16:52:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.804+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c0cc060 con 0x7fb51c06eed0 2024-09-13T16:52:42.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.804+0000 7fb512000700 1 -- 172.21.15.3:0/2942263427 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb508003bb0 con 0x7fb51c06eed0 2024-09-13T16:52:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.804+0000 7fb512000700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb4fc066730 0x7fb4fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.808+0000 7fb512000700 1 -- 172.21.15.3:0/2942263427 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb5080837a0 con 0x7fb51c06eed0 2024-09-13T16:52:42.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.808+0000 7fb519e00700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb4fc066730 0x7fb4fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.808+0000 7fb519e00700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb4fc066730 0x7fb4fc068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009b50 tx=0x7fb50c009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.816+0000 7fb512000700 1 -- 172.21.15.3:0/2942263427 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb5080508e0 con 0x7fb51c06eed0 2024-09-13T16:52:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.996+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb51c15e570 con 0x7fb51c06eed0 2024-09-13T16:52:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:42.996+0000 7fb512000700 1 -- 172.21.15.3:0/2942263427 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb508003e60 con 0x7fb51c06eed0 2024-09-13T16:52:43.002 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:52:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb4fc066730 msgr2=0x7fb4fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb4fc066730 0x7fb4fc068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009b50 tx=0x7fb50c009240 comp rx=0 tx=0).stop 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 msgr2=0x7fb51c0c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0c6640 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c4a0 tx=0x7fb50800c7b0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 shutdown_connections 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb4fc066730 0x7fb4fc068bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb51c15aca0 0x7fb51c0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb51c06eed0 0x7fb51c0c6640 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 --2- 172.21.15.3:0/2942263427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb51c06ea90 0x7fb51c157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 >> 172.21.15.3:0/2942263427 conn(0x7fb51c0b24c0 msgr2=0x7fb51c0cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 shutdown_connections 2024-09-13T16:52:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.000+0000 7fb51b200700 1 -- 172.21.15.3:0/2942263427 wait complete. 2024-09-13T16:52:43.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:52:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.200+0000 7f1a77200700 1 Processor -- start 2024-09-13T16:52:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.200+0000 7f1a77200700 1 -- start start 2024-09-13T16:52:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a77200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a7815aca0 0x7f1a780cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a77200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a7806ea90 0x7f1a780ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a77200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a77200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a780cc5a0 con 0x7f1a7815aca0 2024-09-13T16:52:43.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a77200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a7806dd60 con 0x7f1a7806ea90 2024-09-13T16:52:43.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a77200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a780cf330 con 0x7f1a7806ef00 2024-09-13T16:52:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a7815aca0 0x7f1a780cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a7815aca0 0x7f1a780cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43916/0 (socket says 172.21.15.3:43916) 2024-09-13T16:52:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75e00700 1 -- 172.21.15.3:0/91217857 learned_addr learned my addr 172.21.15.3:0/91217857 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75400700 1 --2- 172.21.15.3:0/91217857 >> v2:172.21.15.157:3300/0 conn(0x7f1a7806ea90 0x7f1a780ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a76800700 1 --2- 172.21.15.3:0/91217857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75e00700 1 -- 172.21.15.3:0/91217857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 msgr2=0x7f1a780cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75e00700 1 --2- 172.21.15.3:0/91217857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75e00700 1 -- 172.21.15.3:0/91217857 >> v2:172.21.15.157:3300/0 conn(0x7f1a7806ea90 msgr2=0x7f1a780ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75e00700 1 --2- 172.21.15.3:0/91217857 >> v2:172.21.15.157:3300/0 conn(0x7f1a7806ea90 0x7f1a780ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75e00700 1 -- 172.21.15.3:0/91217857 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a780b3250 con 0x7f1a7815aca0 2024-09-13T16:52:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.204+0000 7f1a75400700 1 --2- 172.21.15.3:0/91217857 >> v2:172.21.15.157:3300/0 conn(0x7f1a7806ea90 0x7f1a780ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a75e00700 1 --2- 172.21.15.3:0/91217857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780cc060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1a78072ac0 tx=0x7f1a68009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129b91c3fc8fc365 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a74a00700 1 -- 172.21.15.3:0/91217857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68011830 con 0x7f1a7815aca0 2024-09-13T16:52:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a74a00700 1 -- 172.21.15.3:0/91217857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a68011990 con 0x7f1a7815aca0 2024-09-13T16:52:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a74a00700 1 -- 172.21.15.3:0/91217857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68011d20 con 0x7f1a7815aca0 2024-09-13T16:52:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 -- 172.21.15.3:0/91217857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 msgr2=0x7f1a780cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 --2- 172.21.15.3:0/91217857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780cc060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1a78072ac0 tx=0x7f1a68009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 -- 172.21.15.3:0/91217857 shutdown_connections 2024-09-13T16:52:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 --2- 172.21.15.3:0/91217857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 --2- 172.21.15.3:0/91217857 >> v2:172.21.15.157:3300/0 conn(0x7f1a7806ea90 0x7f1a780ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 --2- 172.21.15.3:0/91217857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780cc060 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 -- 172.21.15.3:0/91217857 >> 172.21.15.3:0/91217857 conn(0x7f1a780b24c0 msgr2=0x7f1a780b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 -- 172.21.15.3:0/91217857 shutdown_connections 2024-09-13T16:52:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.208+0000 7f1a77200700 1 -- 172.21.15.3:0/91217857 wait complete. 2024-09-13T16:52:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a77200700 1 Processor -- start 2024-09-13T16:52:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a77200700 1 -- start start 2024-09-13T16:52:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a77200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a7806ea90 0x7f1a78159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a77200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a77200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a77200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a780c8000 con 0x7f1a7815aca0 2024-09-13T16:52:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a77200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7815a200 con 0x7f1a7806ea90 2024-09-13T16:52:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a77200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a780c7a80 con 0x7f1a7806ef00 2024-09-13T16:52:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a76800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a76800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43920/0 (socket says 172.21.15.3:43920) 2024-09-13T16:52:43.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a76800700 1 -- 172.21.15.3:0/1312142216 learned_addr learned my addr 172.21.15.3:0/1312142216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a75400700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a75e00700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a7806ea90 0x7f1a78159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a76800700 1 -- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 msgr2=0x7f1a780b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a76800700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a76800700 1 -- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a7806ea90 msgr2=0x7f1a78159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a76800700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a7806ea90 0x7f1a78159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a76800700 1 -- 172.21.15.3:0/1312142216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a640094b0 con 0x7f1a7815aca0 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a75400700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.212+0000 7f1a75e00700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a7806ea90 0x7f1a78159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.216+0000 7f1a76800700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780b1740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009fd0 tx=0x7f1a7000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.216+0000 7f1a6e000700 1 -- 172.21.15.3:0/1312142216 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7000ede0 con 0x7f1a7815aca0 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.216+0000 7f1a6e000700 1 -- 172.21.15.3:0/1312142216 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a70014420 con 0x7f1a7815aca0 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.216+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a68009570 con 0x7f1a7815aca0 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.216+0000 7f1a6e000700 1 -- 172.21.15.3:0/1312142216 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a700147d0 con 0x7f1a7815aca0 2024-09-13T16:52:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.216+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7815dbd0 con 0x7f1a7815aca0 2024-09-13T16:52:43.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.216+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7815b080 con 0x7f1a7815aca0 2024-09-13T16:52:43.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.216+0000 7f1a6e000700 1 -- 172.21.15.3:0/1312142216 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1a70014930 con 0x7f1a7815aca0 2024-09-13T16:52:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.220+0000 7f1a6e000700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a58066730 0x7f1a58068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.220+0000 7f1a6e000700 1 -- 172.21.15.3:0/1312142216 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1a70084a00 con 0x7f1a7815aca0 2024-09-13T16:52:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.220+0000 7f1a75e00700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a58066730 0x7f1a58068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.220+0000 7f1a75e00700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a58066730 0x7f1a58068bf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a68000c00 tx=0x7f1a68008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:43.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.224+0000 7f1a6e000700 1 -- 172.21.15.3:0/1312142216 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1a70051bd0 con 0x7f1a7815aca0 2024-09-13T16:52:43.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.412+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a78070500 con 0x7f1a7815aca0 2024-09-13T16:52:43.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.412+0000 7f1a6e000700 1 -- 172.21.15.3:0/1312142216 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a700551f0 con 0x7f1a7815aca0 2024-09-13T16:52:43.418 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a58066730 msgr2=0x7f1a58068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a58066730 0x7f1a58068bf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a68000c00 tx=0x7f1a68008030 comp rx=0 tx=0).stop 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 msgr2=0x7f1a780b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780b1740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009fd0 tx=0x7f1a7000e3c0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 shutdown_connections 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1a58066730 0x7f1a58068bf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a7815aca0 0x7f1a780b1740 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7806ef00 0x7f1a780b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 --2- 172.21.15.3:0/1312142216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a7806ea90 0x7f1a78159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 >> 172.21.15.3:0/1312142216 conn(0x7f1a780b24c0 msgr2=0x7f1a7806cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:43.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 shutdown_connections 2024-09-13T16:52:43.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.416+0000 7f1a77200700 1 -- 172.21.15.3:0/1312142216 wait complete. 2024-09-13T16:52:43.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:43.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.632+0000 7f01eb200700 1 Processor -- start 2024-09-13T16:52:43.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.632+0000 7f01eb200700 1 -- start start 2024-09-13T16:52:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.632+0000 7f01eb200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01ec0cc060 0x7f01ec0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.632+0000 7f01eb200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec06dd60 con 0x7f01ec0cedf0 2024-09-13T16:52:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.632+0000 7f01eb200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec06f1a0 con 0x7f01ec15aca0 2024-09-13T16:52:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.632+0000 7f01eb200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0b3250 con 0x7f01ec0cc060 2024-09-13T16:52:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01ea800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f01ec0cedf0 0x7f01ec0cd3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34828/0 (socket says 172.21.15.3:34828) 2024-09-13T16:52:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01ea800700 1 -- 172.21.15.3:0/2466636024 learned_addr learned my addr 172.21.15.3:0/2466636024 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e9400700 1 --2- 172.21.15.3:0/2466636024 >> v2:172.21.15.157:3300/0 conn(0x7f01ec0cc060 0x7f01ec0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025982857 0 0) 0x7f01ec06dd60 con 0x7f01ec0cedf0 2024-09-13T16:52:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d00035d0 con 0x7f01ec0cedf0 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603887662 0 0) 0x7f01ec06f1a0 con 0x7f01ec15aca0 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ec06dd60 con 0x7f01ec15aca0 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3608460993 0 0) 0x7f01d00035d0 con 0x7f01ec0cedf0 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 >> v2:172.21.15.157:3300/0 conn(0x7f01ec0cc060 msgr2=0x7f01ec0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 --2- 172.21.15.3:0/2466636024 >> v2:172.21.15.157:3300/0 conn(0x7f01ec0cc060 0x7f01ec0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 >> v1:172.21.15.157:6789/0 conn(0x7f01ec15aca0 legacy=0x7f01ec06ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0cc7b0 con 0x7f01ec0cedf0 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f01e40019f0 con 0x7f01ec0cedf0 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 >> v1:172.21.15.3:6789/0 conn(0x7f01ec0cedf0 legacy=0x7f01ec0cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01d0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01d0004870 0x7f01d0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0005230 0x7f01d000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b3250 con 0x7f01d0004870 2024-09-13T16:52:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d000c160 con 0x7f01d0003bb0 2024-09-13T16:52:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d000c320 con 0x7f01d0005230 2024-09-13T16:52:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e9e00700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01d0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e9400700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01d0004870 0x7f01d0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01ea800700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0005230 0x7f01d000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e9e00700 1 -- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0005230 msgr2=0x7f01d000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e9e00700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0005230 0x7f01d000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e9e00700 1 -- 172.21.15.3:0/2466636024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01d0004870 msgr2=0x7f01d0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e9e00700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01d0004870 0x7f01d0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01e9e00700 1 -- 172.21.15.3:0/2466636024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d00035d0 con 0x7f01d0003bb0 2024-09-13T16:52:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01ea800700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0005230 0x7f01d000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.636+0000 7f01ea800700 1 -- 172.21.15.3:0/2466636024 reap_dead start 2024-09-13T16:52:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01e9e00700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01d0003fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f01dc001960 tx=0x7f01d00058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49b64f44194407dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc001450 con 0x7f01d0003bb0 2024-09-13T16:52:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc0052f0 con 0x7f01d0003bb0 2024-09-13T16:52:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01e8a00700 1 -- 172.21.15.3:0/2466636024 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc001d30 con 0x7f01d0003bb0 2024-09-13T16:52:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01eb200700 1 -- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 msgr2=0x7f01d0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01eb200700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01d0003fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f01dc001960 tx=0x7f01d00058a0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01eb200700 1 -- 172.21.15.3:0/2466636024 shutdown_connections 2024-09-13T16:52:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01eb200700 1 --2- 172.21.15.3:0/2466636024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01d0003fb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01eb200700 1 -- 172.21.15.3:0/2466636024 >> 172.21.15.3:0/2466636024 conn(0x7f01ec0b24c0 msgr2=0x7f01ec0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01eb200700 1 -- 172.21.15.3:0/2466636024 shutdown_connections 2024-09-13T16:52:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.640+0000 7f01eb200700 1 -- 172.21.15.3:0/2466636024 wait complete. 2024-09-13T16:52:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 Processor -- start 2024-09-13T16:52:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 -- start start 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01ec0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0c50d0 0x7f01ec0c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01ec0c5a70 0x7f01ec15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec15d0b0 con 0x7f01ec0c5a70 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec159eb0 con 0x7f01d0003bb0 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec15ca50 con 0x7f01ec0c50d0 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01ec0c5a70 0x7f01ec15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01ec0c5a70 0x7f01ec15d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43936/0 (socket says 172.21.15.3:43936) 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 -- 172.21.15.3:0/2893471213 learned_addr learned my addr 172.21.15.3:0/2893471213 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01e9400700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0c50d0 0x7f01ec0c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01e9e00700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01ec0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 -- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0c50d0 msgr2=0x7f01ec0c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0c50d0 0x7f01ec0c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 -- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 msgr2=0x7f01ec0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01ec0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 -- 172.21.15.3:0/2893471213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec15b080 con 0x7f01ec0c5a70 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01ea800700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01ec0c5a70 0x7f01ec15d4d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0c7a10 tx=0x7f01e4007290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01e2000700 1 -- 172.21.15.3:0/2893471213 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e400ae60 con 0x7f01ec0c5a70 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01e2000700 1 -- 172.21.15.3:0/2893471213 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40079a0 con 0x7f01ec0c5a70 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec146aa0 con 0x7f01ec0c5a70 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.644+0000 7f01e2000700 1 -- 172.21.15.3:0/2893471213 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40133f0 con 0x7f01ec0c5a70 2024-09-13T16:52:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.648+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec15ccd0 con 0x7f01ec0c5a70 2024-09-13T16:52:43.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.648+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec15b560 con 0x7f01ec0c5a70 2024-09-13T16:52:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.648+0000 7f01e2000700 1 -- 172.21.15.3:0/2893471213 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f01e4013550 con 0x7f01ec0c5a70 2024-09-13T16:52:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.652+0000 7f01e2000700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f01cc066730 0x7f01cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.652+0000 7f01e2000700 1 -- 172.21.15.3:0/2893471213 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f01e40847f0 con 0x7f01ec0c5a70 2024-09-13T16:52:43.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.652+0000 7f01e9e00700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f01cc066730 0x7f01cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:43.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.652+0000 7f01e9e00700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f01cc066730 0x7f01cc068bf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f01dc005040 tx=0x7f01dc00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.656+0000 7f01e2000700 1 -- 172.21.15.3:0/2893471213 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f01ec15b560 con 0x7f01ec0c5a70 2024-09-13T16:52:43.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.888+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01ec15b560 con 0x7f01ec0c5a70 2024-09-13T16:52:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.888+0000 7f01e2000700 1 -- 172.21.15.3:0/2893471213 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f01ec15b560 con 0x7f01ec0c5a70 2024-09-13T16:52:43.894 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:43.894 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:15.362813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5569":{"gid":5569,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/1311223400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1311223400},{"type":"v1","addr":"172.21.15.3:6835","nonce":1311223400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:43.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f01cc066730 msgr2=0x7f01cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f01cc066730 0x7f01cc068bf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f01dc005040 tx=0x7f01dc00a000 comp rx=0 tx=0).stop 2024-09-13T16:52:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01ec0c5a70 msgr2=0x7f01ec15d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01ec0c5a70 0x7f01ec15d4d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0c7a10 tx=0x7f01e4007290 comp rx=0 tx=0).stop 2024-09-13T16:52:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 shutdown_connections 2024-09-13T16:52:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f01cc066730 0x7f01cc068bf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01ec0c5a70 0x7f01ec15d4d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0c50d0 0x7f01ec0c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 --2- 172.21.15.3:0/2893471213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0003bb0 0x7f01ec0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 >> 172.21.15.3:0/2893471213 conn(0x7f01ec0b24c0 msgr2=0x7f01ec0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 shutdown_connections 2024-09-13T16:52:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:43.892+0000 7f01eb200700 1 -- 172.21.15.3:0/2893471213 wait complete. 2024-09-13T16:52:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-09-13T16:52:43.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b9e000700 1 Processor -- start 2024-09-13T16:52:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b9e000700 1 -- start start 2024-09-13T16:52:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b9e000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b98070dc0 0x7f6b9806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b9e000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b9806c3a0 0x7f6b981646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b9e000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b980ce5d0 con 0x7f6b9806ba80 2024-09-13T16:52:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b9e000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9806ddc0 con 0x7f6b98070dc0 2024-09-13T16:52:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b9e000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9815af00 con 0x7f6b9806c3a0 2024-09-13T16:52:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b97e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b98070dc0 0x7f6b9806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.104+0000 7f6b97e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b98070dc0 0x7f6b9806be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35118/0 (socket says 172.21.15.3:35118) 2024-09-13T16:52:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97e00700 1 -- 172.21.15.3:0/4230343790 learned_addr learned my addr 172.21.15.3:0/4230343790 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9d600700 1 --2- 172.21.15.3:0/4230343790 >> v2:172.21.15.157:3300/0 conn(0x7f6b9806c3a0 0x7f6b981646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97e00700 1 -- 172.21.15.3:0/4230343790 >> v2:172.21.15.157:3300/0 conn(0x7f6b9806c3a0 msgr2=0x7f6b981646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97e00700 1 --2- 172.21.15.3:0/4230343790 >> v2:172.21.15.157:3300/0 conn(0x7f6b9806c3a0 0x7f6b981646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97e00700 1 -- 172.21.15.3:0/4230343790 >> v1:172.21.15.157:6789/0 conn(0x7f6b9806ba80 legacy=0x7f6b9806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97e00700 1 -- 172.21.15.3:0/4230343790 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b980cd9c0 con 0x7f6b98070dc0 2024-09-13T16:52:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97e00700 1 --2- 172.21.15.3:0/4230343790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b9806be60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6b98072210 tx=0x7f6b84009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b176de2c200f7ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97400700 1 -- 172.21.15.3:0/4230343790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400b8a0 con 0x7f6b98070dc0 2024-09-13T16:52:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97400700 1 -- 172.21.15.3:0/4230343790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8400ba00 con 0x7f6b98070dc0 2024-09-13T16:52:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b97400700 1 -- 172.21.15.3:0/4230343790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400bd90 con 0x7f6b98070dc0 2024-09-13T16:52:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9e000700 1 -- 172.21.15.3:0/4230343790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 msgr2=0x7f6b9806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9e000700 1 --2- 172.21.15.3:0/4230343790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b9806be60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6b98072210 tx=0x7f6b84009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9e000700 1 -- 172.21.15.3:0/4230343790 shutdown_connections 2024-09-13T16:52:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9e000700 1 --2- 172.21.15.3:0/4230343790 >> v2:172.21.15.157:3300/0 conn(0x7f6b9806c3a0 0x7f6b981646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9e000700 1 --2- 172.21.15.3:0/4230343790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b9806be60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9e000700 1 -- 172.21.15.3:0/4230343790 >> 172.21.15.3:0/4230343790 conn(0x7f6b9806a2f0 msgr2=0x7f6b9806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9e000700 1 -- 172.21.15.3:0/4230343790 shutdown_connections 2024-09-13T16:52:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.108+0000 7f6b9e000700 1 -- 172.21.15.3:0/4230343790 wait complete. 2024-09-13T16:52:44.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.112+0000 7f6b9e000700 1 Processor -- start 2024-09-13T16:52:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 -- start start 2024-09-13T16:52:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9806ba80 0x7f6b98154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9806c3a0 0x7f6b980abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b980abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981569b0 con 0x7f6b98070dc0 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98155910 con 0x7f6b9806c3a0 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98155bf0 con 0x7f6b9806ba80 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b980abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b980abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35124/0 (socket says 172.21.15.3:35124) 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 -- 172.21.15.3:0/3041452519 learned_addr learned my addr 172.21.15.3:0/3041452519 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9cc00700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9806ba80 0x7f6b98154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b97e00700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9806c3a0 0x7f6b980abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 -- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9806ba80 msgr2=0x7f6b98154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9806ba80 0x7f6b98154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 -- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9806c3a0 msgr2=0x7f6b980abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9806c3a0 0x7f6b980abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 -- 172.21.15.3:0/3041452519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84009570 con 0x7f6b98070dc0 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9d600700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b980abff0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007d90 tx=0x7f6b9000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b94c00700 1 -- 172.21.15.3:0/3041452519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000ed30 con 0x7f6b98070dc0 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b94c00700 1 -- 172.21.15.3:0/3041452519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b90014420 con 0x7f6b98070dc0 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b98155ed0 con 0x7f6b98070dc0 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b94c00700 1 -- 172.21.15.3:0/3041452519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b900147d0 con 0x7f6b98070dc0 2024-09-13T16:52:44.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.116+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b98156490 con 0x7f6b98070dc0 2024-09-13T16:52:44.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.120+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b980cd5e0 con 0x7f6b98070dc0 2024-09-13T16:52:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.120+0000 7f6b94c00700 1 -- 172.21.15.3:0/3041452519 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b90014930 con 0x7f6b98070dc0 2024-09-13T16:52:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.124+0000 7f6b94c00700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b78066760 0x7f6b78068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.124+0000 7f6b94c00700 1 -- 172.21.15.3:0/3041452519 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f6b900848e0 con 0x7f6b98070dc0 2024-09-13T16:52:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.124+0000 7f6b9cc00700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b78066760 0x7f6b78068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.124+0000 7f6b9cc00700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b78066760 0x7f6b78068c20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0018a0 tx=0x7f6b8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.132+0000 7f6b94c00700 1 -- 172.21.15.3:0/3041452519 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b90051ab0 con 0x7f6b98070dc0 2024-09-13T16:52:44.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.328+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b980afa80 con 0x7f6b98070dc0 2024-09-13T16:52:44.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.328+0000 7f6b94c00700 1 -- 172.21.15.3:0/3041452519 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16900 (secure 0 0 0) 0x7f6b900550d0 con 0x7f6b98070dc0 2024-09-13T16:52:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:44.336 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":46,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:15.362510+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:01.009263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:02.059017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/750657669":"2024-09-14T16:52:11.310659+0000","172.21.15.3:6834/750657669":"2024-09-14T16:52:11.310659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:44.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b78066760 msgr2=0x7f6b78068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b78066760 0x7f6b78068c20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0018a0 tx=0x7f6b8c009040 comp rx=0 tx=0).stop 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 msgr2=0x7f6b980abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b980abff0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007d90 tx=0x7f6b9000e3c0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 shutdown_connections 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b78066760 0x7f6b78068c20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b98070dc0 0x7f6b980abff0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9806c3a0 0x7f6b980abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 --2- 172.21.15.3:0/3041452519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9806ba80 0x7f6b98154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 >> 172.21.15.3:0/3041452519 conn(0x7f6b9806a2f0 msgr2=0x7f6b980ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 shutdown_connections 2024-09-13T16:52:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.332+0000 7f6b9e000700 1 -- 172.21.15.3:0/3041452519 wait complete. 2024-09-13T16:52:44.360 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T16:52:44.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:44.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c42e00700 1 Processor -- start 2024-09-13T16:52:44.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c42e00700 1 -- start start 2024-09-13T16:52:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c42e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c3c165770 0x7f4c3c164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c42e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c3c165110 0x7f4c3c070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c42e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c06ba80 con 0x7f4c3c160f30 2024-09-13T16:52:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c42e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c06fad0 con 0x7f4c3c165110 2024-09-13T16:52:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c42e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c165550 con 0x7f4c3c165770 2024-09-13T16:52:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c41000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c3c165770 0x7f4c3c164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c42400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c3c165110 0x7f4c3c070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c41a00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4c3c160f30 0x7f4c3c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:50156/0 (socket says 172.21.15.3:50156) 2024-09-13T16:52:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.536+0000 7f4c41a00700 1 -- 172.21.15.3:0/4221994229 learned_addr learned my addr 172.21.15.3:0/4221994229 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:44.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42400700 1 -- 172.21.15.3:0/4221994229 >> v2:172.21.15.157:3300/0 conn(0x7f4c3c165770 msgr2=0x7f4c3c164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:44.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42400700 1 --2- 172.21.15.3:0/4221994229 >> v2:172.21.15.157:3300/0 conn(0x7f4c3c165770 0x7f4c3c164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42400700 1 -- 172.21.15.3:0/4221994229 >> v1:172.21.15.157:6789/0 conn(0x7f4c3c160f30 legacy=0x7f4c3c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42400700 1 -- 172.21.15.3:0/4221994229 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c06bcd0 con 0x7f4c3c165110 2024-09-13T16:52:44.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42400700 1 --2- 172.21.15.3:0/4221994229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c070240 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0b3810 tx=0x7f4c3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c516f1c4b65ccaf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c3be00700 1 -- 172.21.15.3:0/4221994229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400e070 con 0x7f4c3c165110 2024-09-13T16:52:44.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c3be00700 1 -- 172.21.15.3:0/4221994229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400bb20 con 0x7f4c3c165110 2024-09-13T16:52:44.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c3be00700 1 -- 172.21.15.3:0/4221994229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34012430 con 0x7f4c3c165110 2024-09-13T16:52:44.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42e00700 1 -- 172.21.15.3:0/4221994229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 msgr2=0x7f4c3c070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42e00700 1 --2- 172.21.15.3:0/4221994229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c070240 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0b3810 tx=0x7f4c3400b330 comp rx=0 tx=0).stop 2024-09-13T16:52:44.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42e00700 1 -- 172.21.15.3:0/4221994229 shutdown_connections 2024-09-13T16:52:44.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42e00700 1 --2- 172.21.15.3:0/4221994229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c070240 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42e00700 1 --2- 172.21.15.3:0/4221994229 >> v2:172.21.15.157:3300/0 conn(0x7f4c3c165770 0x7f4c3c164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42e00700 1 -- 172.21.15.3:0/4221994229 >> 172.21.15.3:0/4221994229 conn(0x7f4c3c06a2f0 msgr2=0x7f4c3c06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:44.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42e00700 1 -- 172.21.15.3:0/4221994229 shutdown_connections 2024-09-13T16:52:44.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.540+0000 7f4c42e00700 1 -- 172.21.15.3:0/4221994229 wait complete. 2024-09-13T16:52:44.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.544+0000 7f4c42e00700 1 Processor -- start 2024-09-13T16:52:44.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.544+0000 7f4c42e00700 1 -- start start 2024-09-13T16:52:44.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c3c160f30 0x7f4c3c14ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c165770 0x7f4c3c15fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c14d3f0 con 0x7f4c3c165110 2024-09-13T16:52:44.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c14d790 con 0x7f4c3c165770 2024-09-13T16:52:44.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c14da70 con 0x7f4c3c160f30 2024-09-13T16:52:44.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c14e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35140/0 (socket says 172.21.15.3:35140) 2024-09-13T16:52:44.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 -- 172.21.15.3:0/3525850400 learned_addr learned my addr 172.21.15.3:0/3525850400 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:44.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41a00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c3c160f30 0x7f4c3c14ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42400700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c165770 0x7f4c3c15fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 -- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c3c160f30 msgr2=0x7f4c3c14ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c3c160f30 0x7f4c3c14ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 -- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c165770 msgr2=0x7f4c3c15fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c165770 0x7f4c3c15fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 -- 172.21.15.3:0/3525850400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009a70 con 0x7f4c3c165110 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42400700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c165770 0x7f4c3c15fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c41000700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c14e830 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800e840 tx=0x7f4c2800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c39600700 1 -- 172.21.15.3:0/3525850400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c280095a0 con 0x7f4c3c165110 2024-09-13T16:52:44.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c39600700 1 -- 172.21.15.3:0/3525850400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2800c690 con 0x7f4c3c165110 2024-09-13T16:52:44.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c39600700 1 -- 172.21.15.3:0/3525850400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800ca00 con 0x7f4c3c165110 2024-09-13T16:52:44.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c14dd50 con 0x7f4c3c165110 2024-09-13T16:52:44.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c14e310 con 0x7f4c3c165110 2024-09-13T16:52:44.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.548+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c164bd0 con 0x7f4c3c165110 2024-09-13T16:52:44.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.552+0000 7f4c39600700 1 -- 172.21.15.3:0/3525850400 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c2800cb60 con 0x7f4c3c165110 2024-09-13T16:52:44.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.552+0000 7f4c39600700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4c1c066760 0x7f4c1c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.552+0000 7f4c39600700 1 -- 172.21.15.3:0/3525850400 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f4c28084340 con 0x7f4c3c165110 2024-09-13T16:52:44.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.552+0000 7f4c41a00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4c1c066760 0x7f4c1c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.556+0000 7f4c41a00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4c1c066760 0x7f4c1c068c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c30005930 tx=0x7f4c30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.560+0000 7f4c39600700 1 -- 172.21.15.3:0/3525850400 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c28051480 con 0x7f4c3c165110 2024-09-13T16:52:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.795+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c3c164940 con 0x7f4c3c165110 2024-09-13T16:52:44.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c39600700 1 -- 172.21.15.3:0/3525850400 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f4c28054aa0 con 0x7f4c3c165110 2024-09-13T16:52:44.804 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:44.805 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:15.362813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5569":{"gid":5569,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/1311223400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1311223400},{"type":"v1","addr":"172.21.15.3:6835","nonce":1311223400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4c1c066760 msgr2=0x7f4c1c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4c1c066760 0x7f4c1c068c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c30005930 tx=0x7f4c30009040 comp rx=0 tx=0).stop 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 msgr2=0x7f4c3c14e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c14e830 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800e840 tx=0x7f4c2800ec00 comp rx=0 tx=0).stop 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 shutdown_connections 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4c1c066760 0x7f4c1c068c20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c165770 0x7f4c3c15fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c165110 0x7f4c3c14e830 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 --2- 172.21.15.3:0/3525850400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c3c160f30 0x7f4c3c14ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 >> 172.21.15.3:0/3525850400 conn(0x7f4c3c06a2f0 msgr2=0x7f4c3c0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:44.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 shutdown_connections 2024-09-13T16:52:44.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.799+0000 7f4c42e00700 1 -- 172.21.15.3:0/3525850400 wait complete. 2024-09-13T16:52:44.810 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-09-13T16:52:44.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.999+0000 7f5362c00700 1 Processor -- start 2024-09-13T16:52:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:44.999+0000 7f5362c00700 1 -- start start 2024-09-13T16:52:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5362c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c070dc0 0x7f535c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5362c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f535c06c3a0 0x7f535c1646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5362c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0ce5d0 con 0x7f535c06ba80 2024-09-13T16:52:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5362c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c06ddc0 con 0x7f535c06c3a0 2024-09-13T16:52:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5362c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15af00 con 0x7f535c070dc0 2024-09-13T16:52:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5360e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c070dc0 0x7f535c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5361800700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f535c06ba80 0x7f535c06d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:50160/0 (socket says 172.21.15.3:50160) 2024-09-13T16:52:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5361800700 1 -- 172.21.15.3:0/2568340080 learned_addr learned my addr 172.21.15.3:0/2568340080 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5362200700 1 --2- 172.21.15.3:0/2568340080 >> v2:172.21.15.157:3300/0 conn(0x7f535c06c3a0 0x7f535c1646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f535be00700 1 -- 172.21.15.3:0/2568340080 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181618070 0 0) 0x7f535c0ce5d0 con 0x7f535c06ba80 2024-09-13T16:52:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f535be00700 1 -- 172.21.15.3:0/2568340080 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53440035d0 con 0x7f535c06ba80 2024-09-13T16:52:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5360e00700 1 -- 172.21.15.3:0/2568340080 >> v2:172.21.15.157:3300/0 conn(0x7f535c06c3a0 msgr2=0x7f535c1646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5360e00700 1 --2- 172.21.15.3:0/2568340080 >> v2:172.21.15.157:3300/0 conn(0x7f535c06c3a0 0x7f535c1646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5360e00700 1 -- 172.21.15.3:0/2568340080 >> v1:172.21.15.157:6789/0 conn(0x7f535c06ba80 legacy=0x7f535c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.003+0000 7f5360e00700 1 -- 172.21.15.3:0/2568340080 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0cd9c0 con 0x7f535c070dc0 2024-09-13T16:52:45.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5360e00700 1 --2- 172.21.15.3:0/2568340080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c070dc0 0x7f535c06be60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5348009850 tx=0x7f5348009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a5ff5aa7a76928 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f535be00700 1 -- 172.21.15.3:0/2568340080 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348011830 con 0x7f535c070dc0 2024-09-13T16:52:45.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f535be00700 1 -- 172.21.15.3:0/2568340080 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5348011990 con 0x7f535c070dc0 2024-09-13T16:52:45.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f535be00700 1 -- 172.21.15.3:0/2568340080 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348011d40 con 0x7f535c070dc0 2024-09-13T16:52:45.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5362c00700 1 -- 172.21.15.3:0/2568340080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c070dc0 msgr2=0x7f535c06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5362c00700 1 --2- 172.21.15.3:0/2568340080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c070dc0 0x7f535c06be60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5348009850 tx=0x7f5348009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:45.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5362c00700 1 -- 172.21.15.3:0/2568340080 shutdown_connections 2024-09-13T16:52:45.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5362c00700 1 --2- 172.21.15.3:0/2568340080 >> v2:172.21.15.157:3300/0 conn(0x7f535c06c3a0 0x7f535c1646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5362c00700 1 --2- 172.21.15.3:0/2568340080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c070dc0 0x7f535c06be60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5362c00700 1 -- 172.21.15.3:0/2568340080 >> 172.21.15.3:0/2568340080 conn(0x7f535c06a2f0 msgr2=0x7f535c06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:45.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5362c00700 1 -- 172.21.15.3:0/2568340080 shutdown_connections 2024-09-13T16:52:45.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.007+0000 7f5362c00700 1 -- 172.21.15.3:0/2568340080 wait complete. 2024-09-13T16:52:45.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.011+0000 7f5362c00700 1 Processor -- start 2024-09-13T16:52:45.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.011+0000 7f5362c00700 1 -- start start 2024-09-13T16:52:45.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.011+0000 7f5362c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535c06ba80 0x7f535c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.011+0000 7f5362c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c06c3a0 0x7f535c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.011+0000 7f5362c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535c070dc0 0x7f535c150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.011+0000 7f5362c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15cf10 con 0x7f535c070dc0 2024-09-13T16:52:45.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.011+0000 7f5362c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1580c0 con 0x7f535c06ba80 2024-09-13T16:52:45.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.011+0000 7f5362c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15d1b0 con 0x7f535c06c3a0 2024-09-13T16:52:45.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535c070dc0 0x7f535c150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535c070dc0 0x7f535c150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35142/0 (socket says 172.21.15.3:35142) 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 -- 172.21.15.3:0/2980710426 learned_addr learned my addr 172.21.15.3:0/2980710426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5361800700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535c06ba80 0x7f535c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5360e00700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c06c3a0 0x7f535c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 -- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c06c3a0 msgr2=0x7f535c157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c06c3a0 0x7f535c157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 -- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535c06ba80 msgr2=0x7f535c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535c06ba80 0x7f535c157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 -- 172.21.15.3:0/2980710426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53480094b0 con 0x7f535c070dc0 2024-09-13T16:52:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362200700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535c070dc0 0x7f535c150230 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f535400efc0 tx=0x7f535400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5359600700 1 -- 172.21.15.3:0/2980710426 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400caa0 con 0x7f535c070dc0 2024-09-13T16:52:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5359600700 1 -- 172.21.15.3:0/2980710426 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400f040 con 0x7f535c070dc0 2024-09-13T16:52:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5359600700 1 -- 172.21.15.3:0/2980710426 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400ce40 con 0x7f535c070dc0 2024-09-13T16:52:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c15d490 con 0x7f535c070dc0 2024-09-13T16:52:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c157770 con 0x7f535c070dc0 2024-09-13T16:52:45.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.015+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c15d9b0 con 0x7f535c070dc0 2024-09-13T16:52:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.023+0000 7f5359600700 1 -- 172.21.15.3:0/2980710426 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5354014480 con 0x7f535c070dc0 2024-09-13T16:52:45.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.023+0000 7f5359600700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f533c066730 0x7f533c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.023+0000 7f5359600700 1 -- 172.21.15.3:0/2980710426 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f53540850b0 con 0x7f535c070dc0 2024-09-13T16:52:45.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.023+0000 7f5361800700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f533c066730 0x7f533c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.023+0000 7f5361800700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f533c066730 0x7f533c068bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5350007fd0 tx=0x7f5350009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.027+0000 7f5359600700 1 -- 172.21.15.3:0/2980710426 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5354052400 con 0x7f535c070dc0 2024-09-13T16:52:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.255+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f535c0cd5e0 con 0x7f535c070dc0 2024-09-13T16:52:45.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.255+0000 7f5359600700 1 -- 172.21.15.3:0/2980710426 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f5354018020 con 0x7f535c070dc0 2024-09-13T16:52:45.261 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:45.262 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:52:03.049167+0000","modified":"2024-09-13T16:52:15.362813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5569":{"gid":5569,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/1311223400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1311223400},{"type":"v1","addr":"172.21.15.3:6835","nonce":1311223400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f533c066730 msgr2=0x7f533c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f533c066730 0x7f533c068bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5350007fd0 tx=0x7f5350009040 comp rx=0 tx=0).stop 2024-09-13T16:52:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535c070dc0 msgr2=0x7f535c150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535c070dc0 0x7f535c150230 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f535400efc0 tx=0x7f535400c0c0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 shutdown_connections 2024-09-13T16:52:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f533c066730 0x7f533c068bf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535c070dc0 0x7f535c150230 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f535c06c3a0 0x7f535c157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 --2- 172.21.15.3:0/2980710426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535c06ba80 0x7f535c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 >> 172.21.15.3:0/2980710426 conn(0x7f535c06a2f0 msgr2=0x7f535c06a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 shutdown_connections 2024-09-13T16:52:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.259+0000 7f5362c00700 1 -- 172.21.15.3:0/2980710426 wait complete. 2024-09-13T16:52:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-09-13T16:52:45.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:45.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.459+0000 7fa365600700 1 Processor -- start 2024-09-13T16:52:45.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.459+0000 7fa365600700 1 -- start start 2024-09-13T16:52:45.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa365600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa36006c270 0x7fa360165b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa365600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa360071170 con 0x7fa360071600 2024-09-13T16:52:45.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa365600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa360070560 con 0x7fa3601444c0 2024-09-13T16:52:45.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa365600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3601448a0 con 0x7fa36006c270 2024-09-13T16:52:45.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa35e000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa360071600 0x7fa36006bb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58758/0 (socket says 172.21.15.3:58758) 2024-09-13T16:52:45.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa35e000700 1 -- 172.21.15.3:0/3282391176 learned_addr learned my addr 172.21.15.3:0/3282391176 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:45.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa35f400700 1 --2- 172.21.15.3:0/3282391176 >> v2:172.21.15.157:3300/0 conn(0x7fa36006c270 0x7fa360165b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa35d600700 1 -- 172.21.15.3:0/3282391176 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206165957 0 0) 0x7fa360071170 con 0x7fa360071600 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa35d600700 1 -- 172.21.15.3:0/3282391176 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3440035d0 con 0x7fa360071600 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa35d600700 1 -- 172.21.15.3:0/3282391176 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250177233 0 0) 0x7fa360070560 con 0x7fa3601444c0 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.463+0000 7fa35d600700 1 -- 172.21.15.3:0/3282391176 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa360071170 con 0x7fa3601444c0 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa35f400700 1 -- 172.21.15.3:0/3282391176 >> v1:172.21.15.157:6789/0 conn(0x7fa3601444c0 legacy=0x7fa36015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa35d600700 1 -- 172.21.15.3:0/3282391176 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3687352930 0 0) 0x7fa3440035d0 con 0x7fa360071600 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa35f400700 1 -- 172.21.15.3:0/3282391176 >> v1:172.21.15.3:6789/0 conn(0x7fa360071600 legacy=0x7fa36006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa35f400700 1 -- 172.21.15.3:0/3282391176 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa360070840 con 0x7fa36006c270 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa35f400700 1 --2- 172.21.15.3:0/3282391176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 0x7fa360165b00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa358009d90 tx=0x7fa35800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4adf8a6a089c9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa35d600700 1 -- 172.21.15.3:0/3282391176 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35800e070 con 0x7fa36006c270 2024-09-13T16:52:45.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa35d600700 1 -- 172.21.15.3:0/3282391176 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa358003a40 con 0x7fa36006c270 2024-09-13T16:52:45.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa35d600700 1 -- 172.21.15.3:0/3282391176 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c001670 con 0x7fa36006c270 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa365600700 1 -- 172.21.15.3:0/3282391176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 msgr2=0x7fa360165b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa365600700 1 --2- 172.21.15.3:0/3282391176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 0x7fa360165b00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa358009d90 tx=0x7fa35800b330 comp rx=0 tx=0).stop 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa365600700 1 -- 172.21.15.3:0/3282391176 shutdown_connections 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa365600700 1 --2- 172.21.15.3:0/3282391176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 0x7fa360165b00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa365600700 1 -- 172.21.15.3:0/3282391176 >> 172.21.15.3:0/3282391176 conn(0x7fa3600b07a0 msgr2=0x7fa3600b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa365600700 1 -- 172.21.15.3:0/3282391176 shutdown_connections 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.467+0000 7fa365600700 1 -- 172.21.15.3:0/3282391176 wait complete. 2024-09-13T16:52:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.471+0000 7fa365600700 1 Processor -- start 2024-09-13T16:52:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.471+0000 7fa365600700 1 -- start start 2024-09-13T16:52:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa365600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 0x7fa36013c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa365600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360071600 0x7fa360140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa365600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3601444c0 0x7fa3601f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa365600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360156b60 con 0x7fa360071600 2024-09-13T16:52:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa365600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360140da0 con 0x7fa36006c270 2024-09-13T16:52:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa365600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36013dbe0 con 0x7fa3601444c0 2024-09-13T16:52:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360071600 0x7fa360140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360071600 0x7fa360140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35154/0 (socket says 172.21.15.3:35154) 2024-09-13T16:52:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 -- 172.21.15.3:0/3817225601 learned_addr learned my addr 172.21.15.3:0/3817225601 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35ea00700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 0x7fa36013c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35f400700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3601444c0 0x7fa3601f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 -- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3601444c0 msgr2=0x7fa3601f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3601444c0 0x7fa3601f9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 -- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 msgr2=0x7fa36013c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 0x7fa36013c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 -- 172.21.15.3:0/3817225601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358009a70 con 0x7fa360071600 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa35e000700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360071600 0x7fa360140860 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00c750 tx=0x7fa34c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa356a00700 1 -- 172.21.15.3:0/3817225601 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c004d00 con 0x7fa360071600 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa356a00700 1 -- 172.21.15.3:0/3817225601 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c004e60 con 0x7fa360071600 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36013dec0 con 0x7fa360071600 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa356a00700 1 -- 172.21.15.3:0/3817225601 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0072d0 con 0x7fa360071600 2024-09-13T16:52:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.475+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36013e480 con 0x7fa360071600 2024-09-13T16:52:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.479+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36013e670 con 0x7fa360071600 2024-09-13T16:52:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.479+0000 7fa356a00700 1 -- 172.21.15.3:0/3817225601 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa34c007430 con 0x7fa360071600 2024-09-13T16:52:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.479+0000 7fa356a00700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa340066730 0x7fa340068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.483+0000 7fa356a00700 1 -- 172.21.15.3:0/3817225601 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fa34c083c80 con 0x7fa360071600 2024-09-13T16:52:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.483+0000 7fa35ea00700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa340066730 0x7fa340068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.483+0000 7fa35ea00700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa340066730 0x7fa340068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa350001850 tx=0x7fa350009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.491+0000 7fa356a00700 1 -- 172.21.15.3:0/3817225601 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa34c050dc0 con 0x7fa360071600 2024-09-13T16:52:45.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.687+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa36006bb60 con 0x7fa360071600 2024-09-13T16:52:45.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.691+0000 7fa356a00700 1 -- 172.21.15.3:0/3817225601 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16900 (secure 0 0 0) 0x7fa34c0543e0 con 0x7fa360071600 2024-09-13T16:52:45.697 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":46,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:15.362510+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:01.009263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:02.059017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/750657669":"2024-09-14T16:52:11.310659+0000","172.21.15.3:6834/750657669":"2024-09-14T16:52:11.310659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa340066730 msgr2=0x7fa340068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa340066730 0x7fa340068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa350001850 tx=0x7fa350009040 comp rx=0 tx=0).stop 2024-09-13T16:52:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360071600 msgr2=0x7fa360140860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360071600 0x7fa360140860 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00c750 tx=0x7fa34c00ca60 comp rx=0 tx=0).stop 2024-09-13T16:52:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 shutdown_connections 2024-09-13T16:52:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa340066730 0x7fa340068bf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3601444c0 0x7fa3601f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360071600 0x7fa360140860 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 --2- 172.21.15.3:0/3817225601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa36006c270 0x7fa36013c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 >> 172.21.15.3:0/3817225601 conn(0x7fa3600b07a0 msgr2=0x7fa3600cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 shutdown_connections 2024-09-13T16:52:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.695+0000 7fa365600700 1 -- 172.21.15.3:0/3817225601 wait complete. 2024-09-13T16:52:45.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T16:52:45.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.887+0000 7f90eea00700 1 Processor -- start 2024-09-13T16:52:45.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.887+0000 7f90eea00700 1 -- start start 2024-09-13T16:52:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90eea00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e80b27a0 0x7f90e80b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90eea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e806d710 0x7f90e806dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90eea00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e80b3460 con 0x7f90e8070be0 2024-09-13T16:52:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90eea00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e806cc30 con 0x7f90e80b27a0 2024-09-13T16:52:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90eea00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e806cf10 con 0x7f90e806d710 2024-09-13T16:52:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90ecc00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e80b27a0 0x7f90e80b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90ecc00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e80b27a0 0x7f90e80b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35156/0 (socket says 172.21.15.3:35156) 2024-09-13T16:52:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90ecc00700 1 -- 172.21.15.3:0/3583416286 learned_addr learned my addr 172.21.15.3:0/3583416286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90ee000700 1 --2- 172.21.15.3:0/3583416286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e806d710 0x7f90e806dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90ecc00700 1 -- 172.21.15.3:0/3583416286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e806d710 msgr2=0x7f90e806dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:45.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90ecc00700 1 --2- 172.21.15.3:0/3583416286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e806d710 0x7f90e806dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90ecc00700 1 -- 172.21.15.3:0/3583416286 >> v1:172.21.15.157:6789/0 conn(0x7f90e8070be0 legacy=0x7f90e8070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.891+0000 7f90ecc00700 1 -- 172.21.15.3:0/3583416286 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e806d1f0 con 0x7f90e80b27a0 2024-09-13T16:52:45.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90ecc00700 1 --2- 172.21.15.3:0/3583416286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80b27a0 0x7f90e80b2be0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009910 tx=0x7f90d4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4f56a788478414 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90e7e00700 1 -- 172.21.15.3:0/3583416286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d400ba60 con 0x7f90e80b27a0 2024-09-13T16:52:45.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90e7e00700 1 -- 172.21.15.3:0/3583416286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d400bbc0 con 0x7f90e80b27a0 2024-09-13T16:52:45.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90e7e00700 1 -- 172.21.15.3:0/3583416286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4009e60 con 0x7f90e80b27a0 2024-09-13T16:52:45.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90eea00700 1 -- 172.21.15.3:0/3583416286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80b27a0 msgr2=0x7f90e80b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90eea00700 1 --2- 172.21.15.3:0/3583416286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80b27a0 0x7f90e80b2be0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009910 tx=0x7f90d4009c20 comp rx=0 tx=0).stop 2024-09-13T16:52:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90eea00700 1 -- 172.21.15.3:0/3583416286 shutdown_connections 2024-09-13T16:52:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90eea00700 1 --2- 172.21.15.3:0/3583416286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e806d710 0x7f90e806dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90eea00700 1 --2- 172.21.15.3:0/3583416286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80b27a0 0x7f90e80b2be0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90eea00700 1 -- 172.21.15.3:0/3583416286 >> 172.21.15.3:0/3583416286 conn(0x7f90e806b400 msgr2=0x7f90e806b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90eea00700 1 -- 172.21.15.3:0/3583416286 shutdown_connections 2024-09-13T16:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.895+0000 7f90eea00700 1 -- 172.21.15.3:0/3583416286 wait complete. 2024-09-13T16:52:45.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.899+0000 7f90eea00700 1 Processor -- start 2024-09-13T16:52:45.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.899+0000 7f90eea00700 1 -- start start 2024-09-13T16:52:45.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90eea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e806d710 0x7f90e8150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90eea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8070be0 0x7f90e8145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90eea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e80b27a0 0x7f90e80cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90eea00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8153430 con 0x7f90e806d710 2024-09-13T16:52:45.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90eea00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e81514a0 con 0x7f90e8070be0 2024-09-13T16:52:45.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90eea00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8150be0 con 0x7f90e80b27a0 2024-09-13T16:52:45.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e806d710 0x7f90e8150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e806d710 0x7f90e8150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35162/0 (socket says 172.21.15.3:35162) 2024-09-13T16:52:45.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 -- 172.21.15.3:0/2308852812 learned_addr learned my addr 172.21.15.3:0/2308852812 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:45.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ecc00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8070be0 0x7f90e8145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ee000700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e80b27a0 0x7f90e80cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 -- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e80b27a0 msgr2=0x7f90e80cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e80b27a0 0x7f90e80cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 -- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8070be0 msgr2=0x7f90e8145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8070be0 0x7f90e8145050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 -- 172.21.15.3:0/2308852812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d4009570 con 0x7f90e806d710 2024-09-13T16:52:45.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ee000700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e80b27a0 0x7f90e80cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:45.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90ed600700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e806d710 0x7f90e8150da0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c500 tx=0x7f90dc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90e5600700 1 -- 172.21.15.3:0/2308852812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0052d0 con 0x7f90e806d710 2024-09-13T16:52:45.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90e5600700 1 -- 172.21.15.3:0/2308852812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc005430 con 0x7f90e806d710 2024-09-13T16:52:45.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e8153610 con 0x7f90e806d710 2024-09-13T16:52:45.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90e5600700 1 -- 172.21.15.3:0/2308852812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0055a0 con 0x7f90e806d710 2024-09-13T16:52:45.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.903+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e8153bd0 con 0x7f90e806d710 2024-09-13T16:52:45.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.907+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e81e3260 con 0x7f90e806d710 2024-09-13T16:52:45.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.907+0000 7f90e5600700 1 -- 172.21.15.3:0/2308852812 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90dc0038e0 con 0x7f90e806d710 2024-09-13T16:52:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.907+0000 7f90e5600700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f90c8066730 0x7f90c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.911+0000 7f90e5600700 1 -- 172.21.15.3:0/2308852812 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f90dc083710 con 0x7f90e806d710 2024-09-13T16:52:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.911+0000 7f90ecc00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f90c8066730 0x7f90c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.911+0000 7f90ecc00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f90c8066730 0x7f90c8068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f90d4000c00 tx=0x7f90d401d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:45.915+0000 7f90e5600700 1 -- 172.21.15.3:0/2308852812 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f90e81e3260 con 0x7f90e806d710 2024-09-13T16:52:46.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.135+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f90e81e3260 con 0x7f90e806d710 2024-09-13T16:52:46.143 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-13T16:52:46.135+0000 7f9c4ac00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:52:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.151+0000 7f90e5600700 1 -- 172.21.15.3:0/2308852812 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f90e81e3260 con 0x7f90e806d710 2024-09-13T16:52:46.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f90c8066730 msgr2=0x7f90c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:46.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f90c8066730 0x7f90c8068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f90d4000c00 tx=0x7f90d401d040 comp rx=0 tx=0).stop 2024-09-13T16:52:46.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e806d710 msgr2=0x7f90e8150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:46.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e806d710 0x7f90e8150da0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c500 tx=0x7f90dc00c810 comp rx=0 tx=0).stop 2024-09-13T16:52:46.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 shutdown_connections 2024-09-13T16:52:46.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f90c8066730 0x7f90c8068bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:46.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e80b27a0 0x7f90e80cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:46.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8070be0 0x7f90e8145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:46.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 --2- 172.21.15.3:0/2308852812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e806d710 0x7f90e8150da0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:46.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 >> 172.21.15.3:0/2308852812 conn(0x7f90e806b400 msgr2=0x7f90e8156fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:46.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 shutdown_connections 2024-09-13T16:52:46.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.155+0000 7f90eea00700 1 -- 172.21.15.3:0/2308852812 wait complete. 2024-09-13T16:52:46.163 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T16:52:46.181 INFO:tasks.ceph.mds.c.smithi003.stderr: -8851> 2024-09-13T16:52:16.340+0000 7fda20c00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:52:46.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T16:52:46.245 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-13T16:52:46.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a9800700 1 Processor -- start 2024-09-13T16:52:46.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a9800700 1 -- start start 2024-09-13T16:52:46.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a9800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37a4071810 0x7f37a40aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:46.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a9800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a4070c70 con 0x7f37a40cf3b0 2024-09-13T16:52:46.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a9800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a415cf20 con 0x7f37a415cb40 2024-09-13T16:52:46.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a9800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a40acc20 con 0x7f37a4071810 2024-09-13T16:52:46.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a8e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37a4071810 0x7f37a40aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:46.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a8e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37a4071810 0x7f37a40aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35188/0 (socket says 172.21.15.3:35188) 2024-09-13T16:52:46.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.359+0000 7f37a8e00700 1 -- 172.21.15.3:0/3969143306 learned_addr learned my addr 172.21.15.3:0/3969143306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:46.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a8e00700 1 -- 172.21.15.3:0/3969143306 >> v1:172.21.15.157:6789/0 conn(0x7f37a415cb40 legacy=0x7f37a4160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:46.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a8e00700 1 -- 172.21.15.3:0/3969143306 >> v1:172.21.15.3:6789/0 conn(0x7f37a40cf3b0 legacy=0x7f37a4071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:46.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a8e00700 1 -- 172.21.15.3:0/3969143306 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a40acf00 con 0x7f37a4071810 2024-09-13T16:52:46.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a8e00700 1 --2- 172.21.15.3:0/3969143306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a4071810 0x7f37a40aeb20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f379c009e50 tx=0x7f379c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc29a6cef4a40f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:46.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a2a00700 1 -- 172.21.15.3:0/3969143306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00e070 con 0x7f37a4071810 2024-09-13T16:52:46.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a2a00700 1 -- 172.21.15.3:0/3969143306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00b9c0 con 0x7f37a4071810 2024-09-13T16:52:46.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a2a00700 1 -- 172.21.15.3:0/3969143306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00bd70 con 0x7f37a4071810 2024-09-13T16:52:46.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a9800700 1 -- 172.21.15.3:0/3969143306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a4071810 msgr2=0x7f37a40aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:46.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a9800700 1 --2- 172.21.15.3:0/3969143306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a4071810 0x7f37a40aeb20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f379c009e50 tx=0x7f379c00b330 comp rx=0 tx=0).stop 2024-09-13T16:52:46.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a9800700 1 -- 172.21.15.3:0/3969143306 shutdown_connections 2024-09-13T16:52:46.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a9800700 1 --2- 172.21.15.3:0/3969143306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a4071810 0x7f37a40aeb20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:46.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a9800700 1 -- 172.21.15.3:0/3969143306 >> 172.21.15.3:0/3969143306 conn(0x7f37a406b400 msgr2=0x7f37a406b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:46.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a9800700 1 -- 172.21.15.3:0/3969143306 shutdown_connections 2024-09-13T16:52:46.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.363+0000 7f37a9800700 1 -- 172.21.15.3:0/3969143306 wait complete. 2024-09-13T16:52:46.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.367+0000 7f37a9800700 1 Processor -- start 2024-09-13T16:52:46.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.367+0000 7f37a9800700 1 -- start start 2024-09-13T16:52:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.367+0000 7f37a9800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37a4071810 0x7f37a41f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.367+0000 7f37a9800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37a40cf3b0 0x7f37a40c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.367+0000 7f37a9800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a415cb40 0x7f37a40c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.367+0000 7f37a9800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a41f57a0 con 0x7f37a415cb40 2024-09-13T16:52:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.367+0000 7f37a9800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40c5440 con 0x7f37a4071810 2024-09-13T16:52:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.367+0000 7f37a9800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40c5720 con 0x7f37a40cf3b0 2024-09-13T16:52:46.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a415cb40 0x7f37a40c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:46.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a415cb40 0x7f37a40c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35204/0 (socket says 172.21.15.3:35204) 2024-09-13T16:52:46.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 -- 172.21.15.3:0/2795107580 learned_addr learned my addr 172.21.15.3:0/2795107580 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:46.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a3e00700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37a4071810 0x7f37a41f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:46.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a3400700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37a40cf3b0 0x7f37a40c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 -- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37a40cf3b0 msgr2=0x7f37a40c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37a40cf3b0 0x7f37a40c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 -- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37a4071810 msgr2=0x7f37a41f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37a4071810 0x7f37a41f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 -- 172.21.15.3:0/2795107580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c009b30 con 0x7f37a415cb40 2024-09-13T16:52:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a8e00700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a415cb40 0x7f37a40c64e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f379c012000 tx=0x7f379c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f379be00700 1 -- 172.21.15.3:0/2795107580 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c004800 con 0x7f37a415cb40 2024-09-13T16:52:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f379be00700 1 -- 172.21.15.3:0/2795107580 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c0099b0 con 0x7f37a415cb40 2024-09-13T16:52:46.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f379be00700 1 -- 172.21.15.3:0/2795107580 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c01b670 con 0x7f37a415cb40 2024-09-13T16:52:46.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a4158c00 con 0x7f37a415cb40 2024-09-13T16:52:46.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.371+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a41591c0 con 0x7f37a415cb40 2024-09-13T16:52:46.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.375+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a40aeb20 con 0x7f37a415cb40 2024-09-13T16:52:46.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.375+0000 7f379be00700 1 -- 172.21.15.3:0/2795107580 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f379c01b7d0 con 0x7f37a415cb40 2024-09-13T16:52:46.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.375+0000 7f379be00700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3784066730 0x7f3784068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:46.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.375+0000 7f379be00700 1 -- 172.21.15.3:0/2795107580 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f379c0859a0 con 0x7f37a415cb40 2024-09-13T16:52:46.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.375+0000 7f37a3e00700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3784066730 0x7f3784068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:46.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.379+0000 7f37a3e00700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3784066730 0x7f3784068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3794000f60 tx=0x7f3794009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.383+0000 7f379be00700 1 -- 172.21.15.3:0/2795107580 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f379c052af0 con 0x7f37a415cb40 2024-09-13T16:52:46.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:46.607+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f37a40b10c0 con 0x7f37a415cb40 2024-09-13T16:52:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.155+0000 7f379be00700 1 -- 172.21.15.3:0/2795107580 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f379c056110 con 0x7f37a415cb40 2024-09-13T16:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3784066730 msgr2=0x7f3784068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3784066730 0x7f3784068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3794000f60 tx=0x7f3794009040 comp rx=0 tx=0).stop 2024-09-13T16:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a415cb40 msgr2=0x7f37a40c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a415cb40 0x7f37a40c64e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f379c012000 tx=0x7f379c004af0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 shutdown_connections 2024-09-13T16:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3784066730 0x7f3784068bf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37a415cb40 0x7f37a40c64e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37a40cf3b0 0x7f37a40c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 --2- 172.21.15.3:0/2795107580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37a4071810 0x7f37a41f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 >> 172.21.15.3:0/2795107580 conn(0x7f37a406b400 msgr2=0x7f37a406a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 shutdown_connections 2024-09-13T16:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.159+0000 7f37a9800700 1 -- 172.21.15.3:0/2795107580 wait complete. 2024-09-13T16:52:47.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T16:52:47.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.375+0000 7f0957e00700 1 Processor -- start 2024-09-13T16:52:47.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.375+0000 7f0957e00700 1 -- start start 2024-09-13T16:52:47.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0957e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09580cf710 0x7f09580b3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0957e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09580afd60 0x7f095806dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0957e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09580cc750 con 0x7f095815ada0 2024-09-13T16:52:47.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0957e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580cc140 con 0x7f09580cf710 2024-09-13T16:52:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0957e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09580ce5f0 con 0x7f09580afd60 2024-09-13T16:52:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0956a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f095815ada0 0x7f09580cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58784/0 (socket says 172.21.15.3:58784) 2024-09-13T16:52:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0956a00700 1 -- 172.21.15.3:0/1365703161 learned_addr learned my addr 172.21.15.3:0/1365703161 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0956000700 1 --2- 172.21.15.3:0/1365703161 >> v2:172.21.15.3:3300/0 conn(0x7f09580cf710 0x7f09580b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0957400700 1 --2- 172.21.15.3:0/1365703161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09580afd60 0x7f095806dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0955600700 1 -- 172.21.15.3:0/1365703161 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978720786 0 0) 0x7f09580cc750 con 0x7f095815ada0 2024-09-13T16:52:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0955600700 1 -- 172.21.15.3:0/1365703161 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f093c0035d0 con 0x7f095815ada0 2024-09-13T16:52:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0956000700 1 -- 172.21.15.3:0/1365703161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09580afd60 msgr2=0x7f095806dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0956000700 1 --2- 172.21.15.3:0/1365703161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09580afd60 0x7f095806dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0956000700 1 -- 172.21.15.3:0/1365703161 >> v1:172.21.15.3:6789/0 conn(0x7f095815ada0 legacy=0x7f09580cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0956000700 1 -- 172.21.15.3:0/1365703161 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09580ce8d0 con 0x7f09580cf710 2024-09-13T16:52:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0956000700 1 --2- 172.21.15.3:0/1365703161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580cf710 0x7f09580b3130 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0944009850 tx=0x7f0944009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cd37a78ecf24c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0955600700 1 -- 172.21.15.3:0/1365703161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0944011830 con 0x7f09580cf710 2024-09-13T16:52:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0955600700 1 -- 172.21.15.3:0/1365703161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0944011990 con 0x7f09580cf710 2024-09-13T16:52:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.379+0000 7f0955600700 1 -- 172.21.15.3:0/1365703161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0944011d00 con 0x7f09580cf710 2024-09-13T16:52:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.383+0000 7f0957e00700 1 -- 172.21.15.3:0/1365703161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580cf710 msgr2=0x7f09580b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.383+0000 7f0957e00700 1 --2- 172.21.15.3:0/1365703161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580cf710 0x7f09580b3130 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0944009850 tx=0x7f0944009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.383+0000 7f0957e00700 1 -- 172.21.15.3:0/1365703161 shutdown_connections 2024-09-13T16:52:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.383+0000 7f0957e00700 1 --2- 172.21.15.3:0/1365703161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09580afd60 0x7f095806dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.383+0000 7f0957e00700 1 --2- 172.21.15.3:0/1365703161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580cf710 0x7f09580b3130 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.383+0000 7f0957e00700 1 -- 172.21.15.3:0/1365703161 >> 172.21.15.3:0/1365703161 conn(0x7f09580b2610 msgr2=0x7f09580b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.383+0000 7f0957e00700 1 -- 172.21.15.3:0/1365703161 shutdown_connections 2024-09-13T16:52:47.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.383+0000 7f0957e00700 1 -- 172.21.15.3:0/1365703161 wait complete. 2024-09-13T16:52:47.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957e00700 1 Processor -- start 2024-09-13T16:52:47.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957e00700 1 -- start start 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580afd60 0x7f0958158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09580cf710 0x7f09580b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f095815ada0 0x7f09581f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c65c0 con 0x7f09580afd60 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c59b0 con 0x7f09580cf710 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c5c90 con 0x7f095815ada0 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0956a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580afd60 0x7f0958158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0956a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580afd60 0x7f0958158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35224/0 (socket says 172.21.15.3:35224) 2024-09-13T16:52:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0956a00700 1 -- 172.21.15.3:0/1408965117 learned_addr learned my addr 172.21.15.3:0/1408965117 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0956000700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09580cf710 0x7f09580b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.387+0000 7f0957400700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f095815ada0 0x7f09581f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0956a00700 1 -- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f095815ada0 msgr2=0x7f09581f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0956a00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f095815ada0 0x7f09581f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0956a00700 1 -- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09580cf710 msgr2=0x7f09580b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0956a00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09580cf710 0x7f09580b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0956a00700 1 -- 172.21.15.3:0/1408965117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440094b0 con 0x7f09580afd60 2024-09-13T16:52:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0956a00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580afd60 0x7f0958158ec0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f094800c4d0 tx=0x7f094800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f094f400700 1 -- 172.21.15.3:0/1408965117 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948005d90 con 0x7f09580afd60 2024-09-13T16:52:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f094f400700 1 -- 172.21.15.3:0/1408965117 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094800ce80 con 0x7f09580afd60 2024-09-13T16:52:47.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f094f400700 1 -- 172.21.15.3:0/1408965117 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09480072d0 con 0x7f09580afd60 2024-09-13T16:52:47.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09580c5f70 con 0x7f09580afd60 2024-09-13T16:52:47.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0958159400 con 0x7f09580afd60 2024-09-13T16:52:47.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.391+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09580b0ff0 con 0x7f09580afd60 2024-09-13T16:52:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.395+0000 7f094f400700 1 -- 172.21.15.3:0/1408965117 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0948007430 con 0x7f09580afd60 2024-09-13T16:52:47.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.395+0000 7f094f400700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0938066760 0x7f0938068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.395+0000 7f094f400700 1 -- 172.21.15.3:0/1408965117 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f0948083640 con 0x7f09580afd60 2024-09-13T16:52:47.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.399+0000 7f0956000700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0938066760 0x7f0938068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.399+0000 7f0956000700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0938066760 0x7f0938068c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f09440090b0 tx=0x7f0944008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:47.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.403+0000 7f094f400700 1 -- 172.21.15.3:0/1408965117 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09480507b0 con 0x7f09580afd60 2024-09-13T16:52:47.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:47.587+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f095806dde0 con 0x7f09580afd60 2024-09-13T16:52:48.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.515+0000 7f094f400700 1 -- 172.21.15.3:0/1408965117 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0948053dd0 con 0x7f09580afd60 2024-09-13T16:52:48.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0938066760 msgr2=0x7f0938068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:48.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0938066760 0x7f0938068c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f09440090b0 tx=0x7f0944008030 comp rx=0 tx=0).stop 2024-09-13T16:52:48.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580afd60 msgr2=0x7f0958158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580afd60 0x7f0958158ec0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f094800c4d0 tx=0x7f094800c7e0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 shutdown_connections 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0938066760 0x7f0938068c20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f095815ada0 0x7f09581f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09580cf710 0x7f09580b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 --2- 172.21.15.3:0/1408965117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09580afd60 0x7f0958158ec0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 >> 172.21.15.3:0/1408965117 conn(0x7f09580b2610 msgr2=0x7f09580d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 shutdown_connections 2024-09-13T16:52:48.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.523+0000 7f0957e00700 1 -- 172.21.15.3:0/1408965117 wait complete. 2024-09-13T16:52:48.531 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-13T16:52:48.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T16:52:48.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.731+0000 7fa27ba00700 1 Processor -- start 2024-09-13T16:52:48.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.731+0000 7fa27ba00700 1 -- start start 2024-09-13T16:52:48.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa27ba00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa2740cbe10 0x7fa27406bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa27ba00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2740ce390 0x7fa2740cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa27ba00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa274070290 con 0x7fa27414abb0 2024-09-13T16:52:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa27ba00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2740cce30 con 0x7fa2740ce390 2024-09-13T16:52:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa27ba00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2740cd090 con 0x7fa2740cbe10 2024-09-13T16:52:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2740ce390 0x7fa2740cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2740ce390 0x7fa2740cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35226/0 (socket says 172.21.15.3:35226) 2024-09-13T16:52:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa27414abb0 0x7fa2740cb700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58790/0 (socket says 172.21.15.3:58790) 2024-09-13T16:52:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279a00700 1 -- 172.21.15.3:0/2410894151 learned_addr learned my addr 172.21.15.3:0/2410894151 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa273e00700 1 --2- 172.21.15.3:0/2410894151 >> v2:172.21.15.157:3300/0 conn(0x7fa2740cbe10 0x7fa27406bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279a00700 1 -- 172.21.15.3:0/2410894151 >> v2:172.21.15.157:3300/0 conn(0x7fa2740cbe10 msgr2=0x7fa27406bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279a00700 1 --2- 172.21.15.3:0/2410894151 >> v2:172.21.15.157:3300/0 conn(0x7fa2740cbe10 0x7fa27406bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279a00700 1 -- 172.21.15.3:0/2410894151 >> v1:172.21.15.3:6789/0 conn(0x7fa27414abb0 legacy=0x7fa2740cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279a00700 1 -- 172.21.15.3:0/2410894151 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2740cd370 con 0x7fa2740ce390 2024-09-13T16:52:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa273400700 1 -- 172.21.15.3:0/2410894151 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714186289 0 0) 0x7fa274070290 con 0x7fa27414abb0 2024-09-13T16:52:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa279a00700 1 --2- 172.21.15.3:0/2410894151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2740ce390 0x7fa2740cc8f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa27406de30 tx=0x7fa26c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea64439a0f971b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:48.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa273400700 1 -- 172.21.15.3:0/2410894151 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00e070 con 0x7fa2740ce390 2024-09-13T16:52:48.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa273400700 1 -- 172.21.15.3:0/2410894151 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c00b910 con 0x7fa2740ce390 2024-09-13T16:52:48.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa273400700 1 -- 172.21.15.3:0/2410894151 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00bcc0 con 0x7fa2740ce390 2024-09-13T16:52:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa27ba00700 1 -- 172.21.15.3:0/2410894151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2740ce390 msgr2=0x7fa2740cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.735+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2410894151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2740ce390 0x7fa2740cc8f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa27406de30 tx=0x7fa26c00b330 comp rx=0 tx=0).stop 2024-09-13T16:52:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.739+0000 7fa27ba00700 1 -- 172.21.15.3:0/2410894151 shutdown_connections 2024-09-13T16:52:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.739+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2410894151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2740ce390 0x7fa2740cc8f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.739+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2410894151 >> v2:172.21.15.157:3300/0 conn(0x7fa2740cbe10 0x7fa27406bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.739+0000 7fa27ba00700 1 -- 172.21.15.3:0/2410894151 >> 172.21.15.3:0/2410894151 conn(0x7fa27406a530 msgr2=0x7fa27406a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.739+0000 7fa27ba00700 1 -- 172.21.15.3:0/2410894151 shutdown_connections 2024-09-13T16:52:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.739+0000 7fa27ba00700 1 -- 172.21.15.3:0/2410894151 wait complete. 2024-09-13T16:52:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.739+0000 7fa27ba00700 1 Processor -- start 2024-09-13T16:52:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.739+0000 7fa27ba00700 1 -- start start 2024-09-13T16:52:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa27ba00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2740cbe10 0x7fa2741f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa27ba00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2740ce390 0x7fa2741536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa27ba00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa27414abb0 0x7fa274152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa27ba00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274151650 con 0x7fa27414abb0 2024-09-13T16:52:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa27ba00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741541a0 con 0x7fa2740cbe10 2024-09-13T16:52:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa27ba00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa274154480 con 0x7fa2740ce390 2024-09-13T16:52:48.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa27414abb0 0x7fa274152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa273e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2740ce390 0x7fa2741536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa27414abb0 0x7fa274152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35242/0 (socket says 172.21.15.3:35242) 2024-09-13T16:52:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 -- 172.21.15.3:0/2563568872 learned_addr learned my addr 172.21.15.3:0/2563568872 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279000700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2740cbe10 0x7fa2741f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 -- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2740ce390 msgr2=0x7fa2741536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2740ce390 0x7fa2741536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 -- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2740cbe10 msgr2=0x7fa2741f60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2740cbe10 0x7fa2741f60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 -- 172.21.15.3:0/2563568872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c009b30 con 0x7fa27414abb0 2024-09-13T16:52:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa273e00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2740ce390 0x7fa2741536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa279a00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa27414abb0 0x7fa274152060 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa2740cc250 tx=0x7fa26c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa270c00700 1 -- 172.21.15.3:0/2563568872 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c004840 con 0x7fa27414abb0 2024-09-13T16:52:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa270c00700 1 -- 172.21.15.3:0/2563568872 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c0099b0 con 0x7fa27414abb0 2024-09-13T16:52:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa274151860 con 0x7fa27414abb0 2024-09-13T16:52:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa270c00700 1 -- 172.21.15.3:0/2563568872 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c01c680 con 0x7fa27414abb0 2024-09-13T16:52:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.743+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2741f6630 con 0x7fa27414abb0 2024-09-13T16:52:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.747+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa274070700 con 0x7fa27414abb0 2024-09-13T16:52:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.747+0000 7fa270c00700 1 -- 172.21.15.3:0/2563568872 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa26c01c7e0 con 0x7fa27414abb0 2024-09-13T16:52:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.747+0000 7fa270c00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa254066730 0x7fa254068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.747+0000 7fa270c00700 1 -- 172.21.15.3:0/2563568872 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fa26c085aa0 con 0x7fa27414abb0 2024-09-13T16:52:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.751+0000 7fa279000700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa254066730 0x7fa254068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.751+0000 7fa279000700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa254066730 0x7fa254068bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f60 tx=0x7fa268007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.755+0000 7fa270c00700 1 -- 172.21.15.3:0/2563568872 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa274070700 con 0x7fa27414abb0 2024-09-13T16:52:48.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:48.939+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa274151d80 con 0x7fa27414abb0 2024-09-13T16:52:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.527+0000 7fa270c00700 1 -- 172.21.15.3:0/2563568872 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa274151d80 con 0x7fa27414abb0 2024-09-13T16:52:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa254066730 msgr2=0x7fa254068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa254066730 0x7fa254068bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f60 tx=0x7fa268007e90 comp rx=0 tx=0).stop 2024-09-13T16:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa27414abb0 msgr2=0x7fa274152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa27414abb0 0x7fa274152060 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa2740cc250 tx=0x7fa26c004af0 comp rx=0 tx=0).stop 2024-09-13T16:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 shutdown_connections 2024-09-13T16:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa254066730 0x7fa254068bf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa27414abb0 0x7fa274152060 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2740ce390 0x7fa2741536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 --2- 172.21.15.3:0/2563568872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2740cbe10 0x7fa2741f60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 >> 172.21.15.3:0/2563568872 conn(0x7fa27406a530 msgr2=0x7fa274070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 shutdown_connections 2024-09-13T16:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.535+0000 7fa27ba00700 1 -- 172.21.15.3:0/2563568872 wait complete. 2024-09-13T16:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-13T16:52:49.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:52:49.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.643 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-13T16:52:49.645 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:52:49.646 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-13T16:52:49.647 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-13T16:52:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2024-09-13T16:52:49.649 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:49.650 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:52:49.651 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:52:49.652 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-13T16:52:49.653 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-13T16:52:49.654 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:52:49.655 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:52:49.656 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-09-13T16:52:49.656 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-09-13T16:52:49.656 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:52:49.656 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-09-13T16:52:49.656 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.656 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-09-13T16:52:49.656 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.656 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-09-13T16:52:49.657 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.657 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-09-13T16:52:49.657 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-09-13T16:52:49.657 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.657 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-09-13T16:52:49.657 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-09-13T16:52:49.657 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.657 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:52:49.658 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-09-13T16:52:49.658 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.658 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:52:49.658 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:52:49.658 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.658 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:52:49.659 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-09-13T16:52:49.660 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.661 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-09-13T16:52:49.661 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-09-13T16:52:49.661 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:52:49.661 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.661 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-09-13T16:52:49.662 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-09-13T16:52:49.662 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.662 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-09-13T16:52:49.662 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-09-13T16:52:49.662 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-09-13T16:52:49.663 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.663 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-09-13T16:52:49.663 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-09-13T16:52:49.663 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.663 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:52:49.663 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.664 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-09-13T16:52:49.664 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.664 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-09-13T16:52:49.664 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.664 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:52:49.664 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-09-13T16:52:49.665 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.665 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-09-13T16:52:49.665 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-09-13T16:52:49.665 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-09-13T16:52:49.665 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-09-13T16:52:49.666 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2024-09-13T16:52:49.666 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2024-09-13T16:52:49.666 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:49.666 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-09-13T16:52:49.666 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2024-09-13T16:52:49.666 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:52:49.667 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-09-13T16:52:49.667 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:52:49.667 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:52:49.667 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-09-13T16:52:49.667 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:52:49.668 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.668 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-09-13T16:52:49.668 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:49.668 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-09-13T16:52:49.668 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:52:49.668 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:52:49.669 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:52:49.669 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:52:49.669 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-09-13T16:52:49.669 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:52:49.669 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.670 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-09-13T16:52:49.670 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:49.670 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:52:49.670 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-09-13T16:52:49.670 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:52:49.670 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-09-13T16:52:49.671 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-09-13T16:52:49.671 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:52:49.671 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-09-13T16:52:49.671 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-09-13T16:52:49.671 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:52:49.672 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.672 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:52:49.672 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:52:49.672 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:52:49.672 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.672 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:52:49.673 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:52:49.673 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:52:49.673 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.673 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-09-13T16:52:49.674 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:52:49.674 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.674 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:52:49.674 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-09-13T16:52:49.674 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on pool no app = False 2024-09-13T16:52:49.675 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:52:49.675 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-09-13T16:52:49.675 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-09-13T16:52:49.675 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-09-13T16:52:49.675 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-09-13T16:52:49.675 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:52:49.676 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:52:49.676 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-09-13T16:52:49.676 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:52:49.676 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:49.676 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-09-13T16:52:49.677 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:52:49.677 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:52:49.677 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-09-13T16:52:49.677 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-09-13T16:52:49.677 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-09-13T16:52:49.677 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-09-13T16:52:49.678 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-09-13T16:52:49.678 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-09-13T16:52:49.678 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-09-13T16:52:49.678 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-09-13T16:52:49.678 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-09-13T16:52:49.678 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-09-13T16:52:49.679 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:52:49.679 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-09-13T16:52:49.679 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-09-13T16:52:49.679 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-09-13T16:52:49.679 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-09-13T16:52:49.680 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:52:49.680 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:52:49.680 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:52:49.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T16:52:49.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.867+0000 7f45a9c00700 1 Processor -- start 2024-09-13T16:52:49.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.867+0000 7f45a9c00700 1 -- start start 2024-09-13T16:52:49.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a4159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a406cba0 con 0x7f45a406aaa0 2024-09-13T16:52:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a406ce80 con 0x7f45a41507b0 2024-09-13T16:52:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a406d160 con 0x7f45a40b2ce0 2024-09-13T16:52:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a3400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f45a406aaa0 0x7f45a415af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58796/0 (socket says 172.21.15.3:58796) 2024-09-13T16:52:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a3400700 1 -- 172.21.15.3:0/2154171719 learned_addr learned my addr 172.21.15.3:0/2154171719 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9200700 1 --2- 172.21.15.3:0/2154171719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a4159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a2a00700 1 -- 172.21.15.3:0/2154171719 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780653210 0 0) 0x7f45a406cba0 con 0x7f45a406aaa0 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a2a00700 1 -- 172.21.15.3:0/2154171719 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45880035d0 con 0x7f45a406aaa0 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a2a00700 1 -- 172.21.15.3:0/2154171719 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665123873 0 0) 0x7f45a406ce80 con 0x7f45a41507b0 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a2a00700 1 -- 172.21.15.3:0/2154171719 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45a406cba0 con 0x7f45a41507b0 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9200700 1 -- 172.21.15.3:0/2154171719 >> v1:172.21.15.157:6789/0 conn(0x7f45a41507b0 legacy=0x7f45a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a2a00700 1 -- 172.21.15.3:0/2154171719 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1787885814 0 0) 0x7f45880035d0 con 0x7f45a406aaa0 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9200700 1 -- 172.21.15.3:0/2154171719 >> v1:172.21.15.3:6789/0 conn(0x7f45a406aaa0 legacy=0x7f45a415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9200700 1 -- 172.21.15.3:0/2154171719 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a406d440 con 0x7f45a40b2ce0 2024-09-13T16:52:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a9200700 1 --2- 172.21.15.3:0/2154171719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a4159e20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f459c009dc0 tx=0x7f459c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc92b5e43594a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:49.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a2a00700 1 -- 172.21.15.3:0/2154171719 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c00e070 con 0x7f45a40b2ce0 2024-09-13T16:52:49.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a2a00700 1 -- 172.21.15.3:0/2154171719 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c003a40 con 0x7f45a40b2ce0 2024-09-13T16:52:49.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.871+0000 7f45a2a00700 1 -- 172.21.15.3:0/2154171719 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590001670 con 0x7f45a40b2ce0 2024-09-13T16:52:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 -- 172.21.15.3:0/2154171719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 msgr2=0x7f45a4159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 --2- 172.21.15.3:0/2154171719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a4159e20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f459c009dc0 tx=0x7f459c00b330 comp rx=0 tx=0).stop 2024-09-13T16:52:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 -- 172.21.15.3:0/2154171719 shutdown_connections 2024-09-13T16:52:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 --2- 172.21.15.3:0/2154171719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a4159e20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 -- 172.21.15.3:0/2154171719 >> 172.21.15.3:0/2154171719 conn(0x7f45a40af620 msgr2=0x7f45a40afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 -- 172.21.15.3:0/2154171719 shutdown_connections 2024-09-13T16:52:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 -- 172.21.15.3:0/2154171719 wait complete. 2024-09-13T16:52:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 Processor -- start 2024-09-13T16:52:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.875+0000 7f45a9c00700 1 -- start start 2024-09-13T16:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a406aaa0 0x7f45a40ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a41507b0 0x7f45a40abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40aa4b0 con 0x7f45a41507b0 2024-09-13T16:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40aca40 con 0x7f45a406aaa0 2024-09-13T16:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40aa700 con 0x7f45a40b2ce0 2024-09-13T16:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a41507b0 0x7f45a40abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a41507b0 0x7f45a40abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35252/0 (socket says 172.21.15.3:35252) 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 -- 172.21.15.3:0/616785089 learned_addr learned my addr 172.21.15.3:0/616785089 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a3400700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a3e00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a406aaa0 0x7f45a40ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 -- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 msgr2=0x7f45a414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a414ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 -- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a406aaa0 msgr2=0x7f45a40ab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a406aaa0 0x7f45a40ab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 -- 172.21.15.3:0/616785089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c009a70 con 0x7f45a41507b0 2024-09-13T16:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9200700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a41507b0 0x7f45a40abf80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f459c00b370 tx=0x7f459c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f459be00700 1 -- 172.21.15.3:0/616785089 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0044c0 con 0x7f45a41507b0 2024-09-13T16:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f459be00700 1 -- 172.21.15.3:0/616785089 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c004620 con 0x7f45a41507b0 2024-09-13T16:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a40aa9e0 con 0x7f45a41507b0 2024-09-13T16:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.879+0000 7f459be00700 1 -- 172.21.15.3:0/616785089 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c004990 con 0x7f45a41507b0 2024-09-13T16:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.883+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a40ab720 con 0x7f45a41507b0 2024-09-13T16:52:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.883+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a4159e20 con 0x7f45a41507b0 2024-09-13T16:52:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.883+0000 7f459be00700 1 -- 172.21.15.3:0/616785089 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f459c01d470 con 0x7f45a41507b0 2024-09-13T16:52:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.887+0000 7f459be00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4584066730 0x7f4584068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.887+0000 7f459be00700 1 -- 172.21.15.3:0/616785089 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f459c0858c0 con 0x7f45a41507b0 2024-09-13T16:52:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.887+0000 7f45a3e00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4584066730 0x7f4584068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.887+0000 7f45a3e00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4584066730 0x7f4584068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45940011e0 tx=0x7f4594009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:49.891+0000 7f459be00700 1 -- 172.21.15.3:0/616785089 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f459c056810 con 0x7f45a41507b0 2024-09-13T16:52:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.091+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f45a40b4a60 con 0x7f45a41507b0 2024-09-13T16:52:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.095+0000 7f459be00700 1 -- 172.21.15.3:0/616785089 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f459c0563a0 con 0x7f45a41507b0 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4584066730 msgr2=0x7f4584068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4584066730 0x7f4584068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45940011e0 tx=0x7f4594009040 comp rx=0 tx=0).stop 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a41507b0 msgr2=0x7f45a40abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a41507b0 0x7f45a40abf80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f459c00b370 tx=0x7f459c004af0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 shutdown_connections 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4584066730 0x7f4584068bf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a41507b0 0x7f45a40abf80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45a40b2ce0 0x7f45a414ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 --2- 172.21.15.3:0/616785089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a406aaa0 0x7f45a40ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 >> 172.21.15.3:0/616785089 conn(0x7f45a40af620 msgr2=0x7f45a4070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 shutdown_connections 2024-09-13T16:52:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.099+0000 7f45a9c00700 1 -- 172.21.15.3:0/616785089 wait complete. 2024-09-13T16:52:50.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj' 2024-09-13T16:52:50.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd69000700 1 Processor -- start 2024-09-13T16:52:50.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd69000700 1 -- start start 2024-09-13T16:52:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd69000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd64071670 0x7fdd6415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd69000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6406ca70 0x7fdd6415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd69000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6415af60 con 0x7fdd6406c5e0 2024-09-13T16:52:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd69000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd640b3e60 con 0x7fdd64071670 2024-09-13T16:52:50.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd69000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640b2dc0 con 0x7fdd6406ca70 2024-09-13T16:52:50.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd63400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd64071670 0x7fdd6415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd63e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6406ca70 0x7fdd6415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd62a00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdd6406c5e0 0x7fdd6415d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:50214/0 (socket says 172.21.15.3:50214) 2024-09-13T16:52:50.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd63400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd64071670 0x7fdd6415aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:34124/0 (socket says 172.21.15.3:34124) 2024-09-13T16:52:50.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd63e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6406ca70 0x7fdd6415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:44748/0 (socket says 172.21.15.3:44748) 2024-09-13T16:52:50.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.295+0000 7fdd62a00700 1 -- 172.21.15.3:0/605155424 learned_addr learned my addr 172.21.15.3:0/605155424 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:50.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd62000700 1 -- 172.21.15.3:0/605155424 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073108527 0 0) 0x7fdd6415af60 con 0x7fdd6406c5e0 2024-09-13T16:52:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd62000700 1 -- 172.21.15.3:0/605155424 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd480057e0 con 0x7fdd6406c5e0 2024-09-13T16:52:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd63e00700 1 -- 172.21.15.3:0/605155424 >> v2:172.21.15.157:3300/0 conn(0x7fdd64071670 msgr2=0x7fdd6415aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd63e00700 1 --2- 172.21.15.3:0/605155424 >> v2:172.21.15.157:3300/0 conn(0x7fdd64071670 0x7fdd6415aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd63e00700 1 -- 172.21.15.3:0/605155424 >> v1:172.21.15.157:6789/0 conn(0x7fdd6406c5e0 legacy=0x7fdd6415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd63e00700 1 -- 172.21.15.3:0/605155424 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640b30a0 con 0x7fdd6406ca70 2024-09-13T16:52:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd63e00700 1 --2- 172.21.15.3:0/605155424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6406ca70 0x7fdd6415dca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009d90 tx=0x7fdd5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bcc9dcc1de65551 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd62000700 1 -- 172.21.15.3:0/605155424 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c00e070 con 0x7fdd6406ca70 2024-09-13T16:52:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd62000700 1 -- 172.21.15.3:0/605155424 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c00bb20 con 0x7fdd6406ca70 2024-09-13T16:52:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd62000700 1 -- 172.21.15.3:0/605155424 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c012430 con 0x7fdd6406ca70 2024-09-13T16:52:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd69000700 1 -- 172.21.15.3:0/605155424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6406ca70 msgr2=0x7fdd6415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd69000700 1 --2- 172.21.15.3:0/605155424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6406ca70 0x7fdd6415dca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009d90 tx=0x7fdd5c00b330 comp rx=0 tx=0).stop 2024-09-13T16:52:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd69000700 1 -- 172.21.15.3:0/605155424 shutdown_connections 2024-09-13T16:52:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd69000700 1 --2- 172.21.15.3:0/605155424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6406ca70 0x7fdd6415dca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd69000700 1 --2- 172.21.15.3:0/605155424 >> v2:172.21.15.157:3300/0 conn(0x7fdd64071670 0x7fdd6415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd69000700 1 -- 172.21.15.3:0/605155424 >> 172.21.15.3:0/605155424 conn(0x7fdd640b0cf0 msgr2=0x7fdd640b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd69000700 1 -- 172.21.15.3:0/605155424 shutdown_connections 2024-09-13T16:52:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.299+0000 7fdd69000700 1 -- 172.21.15.3:0/605155424 wait complete. 2024-09-13T16:52:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.303+0000 7fdd69000700 1 Processor -- start 2024-09-13T16:52:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.303+0000 7fdd69000700 1 -- start start 2024-09-13T16:52:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.303+0000 7fdd69000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6406c5e0 0x7fdd64151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.303+0000 7fdd69000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd6406ca70 0x7fdd64141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.303+0000 7fdd69000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64071670 0x7fdd64155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.303+0000 7fdd69000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd641508e0 con 0x7fdd6406ca70 2024-09-13T16:52:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.303+0000 7fdd69000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd64150b30 con 0x7fdd6406c5e0 2024-09-13T16:52:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.303+0000 7fdd69000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd64150e10 con 0x7fdd64071670 2024-09-13T16:52:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd6406ca70 0x7fdd64141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd63400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6406c5e0 0x7fdd64151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd6406ca70 0x7fdd64141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35258/0 (socket says 172.21.15.3:35258) 2024-09-13T16:52:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 -- 172.21.15.3:0/1183515683 learned_addr learned my addr 172.21.15.3:0/1183515683 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd63e00700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64071670 0x7fdd64155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 -- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64071670 msgr2=0x7fdd64155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64071670 0x7fdd64155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 -- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6406c5e0 msgr2=0x7fdd64151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6406c5e0 0x7fdd64151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 -- 172.21.15.3:0/1183515683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c009a70 con 0x7fdd6406ca70 2024-09-13T16:52:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd63400700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6406c5e0 0x7fdd64151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd62a00700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd6406ca70 0x7fdd64141380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdd50001200 tx=0x7fdd5000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd5b400700 1 -- 172.21.15.3:0/1183515683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50005d90 con 0x7fdd6406ca70 2024-09-13T16:52:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd5b400700 1 -- 172.21.15.3:0/1183515683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd500070d0 con 0x7fdd6406ca70 2024-09-13T16:52:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd5b400700 1 -- 172.21.15.3:0/1183515683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50007460 con 0x7fdd6406ca70 2024-09-13T16:52:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd641510f0 con 0x7fdd6406ca70 2024-09-13T16:52:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.307+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd641516b0 con 0x7fdd6406ca70 2024-09-13T16:52:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.311+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd641518d0 con 0x7fdd6406ca70 2024-09-13T16:52:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.311+0000 7fdd5b400700 1 -- 172.21.15.3:0/1183515683 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd500075c0 con 0x7fdd6406ca70 2024-09-13T16:52:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.311+0000 7fdd5b400700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd44066760 0x7fdd44068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.315+0000 7fdd5b400700 1 -- 172.21.15.3:0/1183515683 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fdd500834b0 con 0x7fdd6406ca70 2024-09-13T16:52:50.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.315+0000 7fdd63400700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd44066760 0x7fdd44068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.315+0000 7fdd63400700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd44066760 0x7fdd44068c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009f40 tx=0x7fdd54009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.319+0000 7fdd5b400700 1 -- 172.21.15.3:0/1183515683 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd500508f0 con 0x7fdd6406ca70 2024-09-13T16:52:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:50.575+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]} v 0) v1 -- 0x7fdd6406ce90 con 0x7fdd6406ca70 2024-09-13T16:52:51.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.535+0000 7fdd5b400700 1 -- 172.21.15.3:0/1183515683 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]}]=0 v144) v1 ==== 147+0+0 (secure 0 0 0) 0x7fdd50053f10 con 0x7fdd6406ca70 2024-09-13T16:52:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd44066760 msgr2=0x7fdd44068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd44066760 0x7fdd44068c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009f40 tx=0x7fdd54009240 comp rx=0 tx=0).stop 2024-09-13T16:52:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd6406ca70 msgr2=0x7fdd64141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd6406ca70 0x7fdd64141380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdd50001200 tx=0x7fdd5000c9e0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 shutdown_connections 2024-09-13T16:52:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd44066760 0x7fdd44068c20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64071670 0x7fdd64155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd6406ca70 0x7fdd64141380 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 --2- 172.21.15.3:0/1183515683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6406c5e0 0x7fdd64151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 >> 172.21.15.3:0/1183515683 conn(0x7fdd640b0cf0 msgr2=0x7fdd640706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 shutdown_connections 2024-09-13T16:52:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.539+0000 7fdd69000700 1 -- 172.21.15.3:0/1183515683 wait complete. 2024-09-13T16:52:51.567 INFO:tasks.cephfs_test_runner:The maximum number of openfiles omap objects keys are now equal to ... ok 2024-09-13T16:52:51.568 INFO:tasks.cephfs_test_runner:Starting test: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) Opening a file should increment omap_total_updates by 1. 2024-09-13T16:52:51.568 INFO:tasks.cephfs_test_runner:test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) 2024-09-13T16:52:51.568 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:51.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.751+0000 7fca8ea00700 1 Processor -- start 2024-09-13T16:52:51.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.751+0000 7fca8ea00700 1 -- start start 2024-09-13T16:52:51.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8ea00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca881468f0 0x7fca880b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:51.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8ea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca880b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:51.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8ea00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca880cdfe0 con 0x7fca88146510 2024-09-13T16:52:51.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8ea00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca880b2340 con 0x7fca881468f0 2024-09-13T16:52:51.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8ea00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880b1aa0 con 0x7fca8806be80 2024-09-13T16:52:51.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca880b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:51.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8cc00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca881468f0 0x7fca880b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:51.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca880b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:44762/0 (socket says 172.21.15.3:44762) 2024-09-13T16:52:51.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8e000700 1 -- 172.21.15.3:0/3122964197 learned_addr learned my addr 172.21.15.3:0/3122964197 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:51.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8cc00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca881468f0 0x7fca880b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:34136/0 (socket says 172.21.15.3:34136) 2024-09-13T16:52:51.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.755+0000 7fca8d600700 1 --1- 172.21.15.3:0/3122964197 >> v1:172.21.15.157:6789/0 conn(0x7fca88146510 0x7fca880b1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:50224/0 (socket says 172.21.15.3:50224) 2024-09-13T16:52:51.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8e000700 1 -- 172.21.15.3:0/3122964197 >> v2:172.21.15.157:3300/0 conn(0x7fca881468f0 msgr2=0x7fca880b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:51.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8e000700 1 --2- 172.21.15.3:0/3122964197 >> v2:172.21.15.157:3300/0 conn(0x7fca881468f0 0x7fca880b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8e000700 1 -- 172.21.15.3:0/3122964197 >> v1:172.21.15.157:6789/0 conn(0x7fca88146510 legacy=0x7fca880b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:51.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8e000700 1 -- 172.21.15.3:0/3122964197 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca880b1d80 con 0x7fca8806be80 2024-09-13T16:52:51.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca87e00700 1 -- 172.21.15.3:0/3122964197 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581578394 0 0) 0x7fca880cdfe0 con 0x7fca88146510 2024-09-13T16:52:51.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8cc00700 1 --2- 172.21.15.3:0/3122964197 >> v2:172.21.15.157:3300/0 conn(0x7fca881468f0 0x7fca880b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:51.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8e000700 1 --2- 172.21.15.3:0/3122964197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca880b5010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca80009e50 tx=0x7fca8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6181a3a38934db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:51.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca87e00700 1 -- 172.21.15.3:0/3122964197 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8000e070 con 0x7fca8806be80 2024-09-13T16:52:51.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca87e00700 1 -- 172.21.15.3:0/3122964197 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8000b910 con 0x7fca8806be80 2024-09-13T16:52:51.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca87e00700 1 -- 172.21.15.3:0/3122964197 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8000bcc0 con 0x7fca8806be80 2024-09-13T16:52:51.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8ea00700 1 -- 172.21.15.3:0/3122964197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 msgr2=0x7fca880b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:51.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3122964197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca880b5010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca80009e50 tx=0x7fca8000b330 comp rx=0 tx=0).stop 2024-09-13T16:52:51.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8ea00700 1 -- 172.21.15.3:0/3122964197 shutdown_connections 2024-09-13T16:52:51.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3122964197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca880b5010 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3122964197 >> v2:172.21.15.157:3300/0 conn(0x7fca881468f0 0x7fca880b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8ea00700 1 -- 172.21.15.3:0/3122964197 >> 172.21.15.3:0/3122964197 conn(0x7fca880b29f0 msgr2=0x7fca880b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:51.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8ea00700 1 -- 172.21.15.3:0/3122964197 shutdown_connections 2024-09-13T16:52:51.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.759+0000 7fca8ea00700 1 -- 172.21.15.3:0/3122964197 wait complete. 2024-09-13T16:52:51.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8ea00700 1 Processor -- start 2024-09-13T16:52:51.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8ea00700 1 -- start start 2024-09-13T16:52:51.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8ea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca88141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:51.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88146510 0x7fca8815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:51.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca881468f0 0x7fca880cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:51.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8ea00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca881419a0 con 0x7fca88146510 2024-09-13T16:52:51.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8ea00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca880cbb80 con 0x7fca881468f0 2024-09-13T16:52:51.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8ea00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880cb8a0 con 0x7fca8806be80 2024-09-13T16:52:51.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.763+0000 7fca8cc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88146510 0x7fca8815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:51.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8cc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88146510 0x7fca8815c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35260/0 (socket says 172.21.15.3:35260) 2024-09-13T16:52:51.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8cc00700 1 -- 172.21.15.3:0/3649141035 learned_addr learned my addr 172.21.15.3:0/3649141035 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:51.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca881468f0 0x7fca880cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:51.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8d600700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca88141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:51.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8cc00700 1 -- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 msgr2=0x7fca88141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:51.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8cc00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca88141460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8cc00700 1 -- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca881468f0 msgr2=0x7fca880cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:51.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8cc00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca881468f0 0x7fca880cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:51.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8cc00700 1 -- 172.21.15.3:0/3649141035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740094b0 con 0x7fca88146510 2024-09-13T16:52:51.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8cc00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88146510 0x7fca8815c6a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fca7400e9b0 tx=0x7fca7400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:51.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca85600700 1 -- 172.21.15.3:0/3649141035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7400ee50 con 0x7fca88146510 2024-09-13T16:52:51.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca85600700 1 -- 172.21.15.3:0/3649141035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7400c850 con 0x7fca88146510 2024-09-13T16:52:51.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca80009b30 con 0x7fca88146510 2024-09-13T16:52:51.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca85600700 1 -- 172.21.15.3:0/3649141035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7400cc00 con 0x7fca88146510 2024-09-13T16:52:51.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8806eb80 con 0x7fca88146510 2024-09-13T16:52:51.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.767+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8806e6e0 con 0x7fca88146510 2024-09-13T16:52:51.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.771+0000 7fca85600700 1 -- 172.21.15.3:0/3649141035 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fca74004020 con 0x7fca88146510 2024-09-13T16:52:51.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.771+0000 7fca85600700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fca68066730 0x7fca68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:51.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.771+0000 7fca85600700 1 -- 172.21.15.3:0/3649141035 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fca74083f30 con 0x7fca88146510 2024-09-13T16:52:51.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.771+0000 7fca8d600700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fca68066730 0x7fca68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:51.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.775+0000 7fca8d600700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fca68066730 0x7fca68068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fca7c000f60 tx=0x7fca7c007e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:51.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:51.775+0000 7fca85600700 1 -- 172.21.15.3:0/3649141035 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fca740afdb0 con 0x7fca88146510 2024-09-13T16:52:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.043+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca88073920 con 0x7fca88146510 2024-09-13T16:52:52.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.043+0000 7fca85600700 1 -- 172.21.15.3:0/3649141035 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+13267 (secure 0 0 0) 0x7fca74054f00 con 0x7fca88146510 2024-09-13T16:52:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:52.049 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":49,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:49.519016+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/1311223400":"2024-09-14T16:52:46.140755+0000","172.21.15.3:6832/1311223400":"2024-09-14T16:52:46.140755+0000","172.21.15.3:6835/750657669":"2024-09-14T16:52:11.310659+0000","172.21.15.3:6834/750657669":"2024-09-14T16:52:11.310659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:52.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fca68066730 msgr2=0x7fca68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fca68066730 0x7fca68068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fca7c000f60 tx=0x7fca7c007e00 comp rx=0 tx=0).stop 2024-09-13T16:52:52.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88146510 msgr2=0x7fca8815c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88146510 0x7fca8815c6a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fca7400e9b0 tx=0x7fca7400ecc0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 shutdown_connections 2024-09-13T16:52:52.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fca68066730 0x7fca68068bf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca881468f0 0x7fca880cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88146510 0x7fca8815c6a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 --2- 172.21.15.3:0/3649141035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806be80 0x7fca88141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 >> 172.21.15.3:0/3649141035 conn(0x7fca880b29f0 msgr2=0x7fca880b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:52.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 shutdown_connections 2024-09-13T16:52:52.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.047+0000 7fca8ea00700 1 -- 172.21.15.3:0/3649141035 wait complete. 2024-09-13T16:52:52.075 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:52:52.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:52:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.243+0000 7f8534a00700 1 Processor -- start 2024-09-13T16:52:52.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.243+0000 7f8534a00700 1 -- start start 2024-09-13T16:52:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85300cd7c0 0x7f85300cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f853015b110 0x7f853015b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85300b41e0 con 0x7f85301442c0 2024-09-13T16:52:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853006a390 con 0x7f85300cd7c0 2024-09-13T16:52:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853006b1f0 con 0x7f853015b110 2024-09-13T16:52:52.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852de00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85300cd7c0 0x7f85300cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852de00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85300cd7c0 0x7f85300cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35272/0 (socket says 172.21.15.3:35272) 2024-09-13T16:52:52.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852de00700 1 -- 172.21.15.3:0/1990961619 learned_addr learned my addr 172.21.15.3:0/1990961619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:52.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852f200700 1 --2- 172.21.15.3:0/1990961619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f853015b110 0x7f853015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852de00700 1 -- 172.21.15.3:0/1990961619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f853015b110 msgr2=0x7f853015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852de00700 1 --2- 172.21.15.3:0/1990961619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f853015b110 0x7f853015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852de00700 1 -- 172.21.15.3:0/1990961619 >> v1:172.21.15.157:6789/0 conn(0x7f85301442c0 legacy=0x7f853006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852de00700 1 -- 172.21.15.3:0/1990961619 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85300ce170 con 0x7f85300cd7c0 2024-09-13T16:52:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852f200700 1 --2- 172.21.15.3:0/1990961619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f853015b110 0x7f853015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852de00700 1 --2- 172.21.15.3:0/1990961619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85300cd7c0 0x7f85300cdc30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f851c009910 tx=0x7f851c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187ceb8880a349ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852d400700 1 -- 172.21.15.3:0/1990961619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00ba60 con 0x7f85300cd7c0 2024-09-13T16:52:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852d400700 1 -- 172.21.15.3:0/1990961619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00bbc0 con 0x7f85300cd7c0 2024-09-13T16:52:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f852d400700 1 -- 172.21.15.3:0/1990961619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c009e60 con 0x7f85300cd7c0 2024-09-13T16:52:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 -- 172.21.15.3:0/1990961619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85300cd7c0 msgr2=0x7f85300cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 --2- 172.21.15.3:0/1990961619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85300cd7c0 0x7f85300cdc30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f851c009910 tx=0x7f851c009c20 comp rx=0 tx=0).stop 2024-09-13T16:52:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 -- 172.21.15.3:0/1990961619 shutdown_connections 2024-09-13T16:52:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 --2- 172.21.15.3:0/1990961619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f853015b110 0x7f853015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 --2- 172.21.15.3:0/1990961619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85300cd7c0 0x7f85300cdc30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 -- 172.21.15.3:0/1990961619 >> 172.21.15.3:0/1990961619 conn(0x7f85300af620 msgr2=0x7f85300afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 -- 172.21.15.3:0/1990961619 shutdown_connections 2024-09-13T16:52:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.247+0000 7f8534a00700 1 -- 172.21.15.3:0/1990961619 wait complete. 2024-09-13T16:52:52.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.251+0000 7f8534a00700 1 Processor -- start 2024-09-13T16:52:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.251+0000 7f8534a00700 1 -- start start 2024-09-13T16:52:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.251+0000 7f8534a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85300cd7c0 0x7f853015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.251+0000 7f8534a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85301442c0 0x7f85300ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.251+0000 7f8534a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853015b110 0x7f853014f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.251+0000 7f8534a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853013b800 con 0x7f85301442c0 2024-09-13T16:52:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.251+0000 7f8534a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85300adb10 con 0x7f853015b110 2024-09-13T16:52:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.251+0000 7f8534a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85300addf0 con 0x7f85300cd7c0 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85301442c0 0x7f85300ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85301442c0 0x7f85300ad270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35274/0 (socket says 172.21.15.3:35274) 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 -- 172.21.15.3:0/2868230751 learned_addr learned my addr 172.21.15.3:0/2868230751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852e800700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85300cd7c0 0x7f853015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852f200700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853015b110 0x7f853014f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 -- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85300cd7c0 msgr2=0x7f853015a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85300cd7c0 0x7f853015a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 -- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853015b110 msgr2=0x7f853014f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853015b110 0x7f853014f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 -- 172.21.15.3:0/2868230751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009570 con 0x7f85301442c0 2024-09-13T16:52:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f852de00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85301442c0 0x7f85300ad270 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f851c000c00 tx=0x7f851c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f8526a00700 1 -- 172.21.15.3:0/2868230751 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c004610 con 0x7f85301442c0 2024-09-13T16:52:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f8526a00700 1 -- 172.21.15.3:0/2868230751 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c01e070 con 0x7f85301442c0 2024-09-13T16:52:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f8526a00700 1 -- 172.21.15.3:0/2868230751 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c003900 con 0x7f85301442c0 2024-09-13T16:52:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85300ae0d0 con 0x7f85301442c0 2024-09-13T16:52:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.255+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853015a560 con 0x7f85301442c0 2024-09-13T16:52:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.259+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853013bce0 con 0x7f85301442c0 2024-09-13T16:52:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.259+0000 7f8526a00700 1 -- 172.21.15.3:0/2868230751 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f851c003a60 con 0x7f85301442c0 2024-09-13T16:52:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.259+0000 7f8526a00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8510066730 0x7f8510068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.259+0000 7f8526a00700 1 -- 172.21.15.3:0/2868230751 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f851c020070 con 0x7f85301442c0 2024-09-13T16:52:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.259+0000 7f852e800700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8510066730 0x7f8510068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.263+0000 7f852e800700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8510066730 0x7f8510068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f85300cb070 tx=0x7f8520009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.267+0000 7f8526a00700 1 -- 172.21.15.3:0/2868230751 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f853013bce0 con 0x7f85301442c0 2024-09-13T16:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.447+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f853013bce0 con 0x7f85301442c0 2024-09-13T16:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8526a00700 1 -- 172.21.15.3:0/2868230751 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f853013bce0 con 0x7f85301442c0 2024-09-13T16:52:52.456 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:52:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8510066730 msgr2=0x7f8510068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8510066730 0x7f8510068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f85300cb070 tx=0x7f8520009040 comp rx=0 tx=0).stop 2024-09-13T16:52:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85301442c0 msgr2=0x7f85300ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85301442c0 0x7f85300ad270 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f851c000c00 tx=0x7f851c004770 comp rx=0 tx=0).stop 2024-09-13T16:52:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 shutdown_connections 2024-09-13T16:52:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8510066730 0x7f8510068bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853015b110 0x7f853014f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85301442c0 0x7f85300ad270 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 --2- 172.21.15.3:0/2868230751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85300cd7c0 0x7f853015a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 >> 172.21.15.3:0/2868230751 conn(0x7f85300af620 msgr2=0x7f853006b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 shutdown_connections 2024-09-13T16:52:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.451+0000 7f8534a00700 1 -- 172.21.15.3:0/2868230751 wait complete. 2024-09-13T16:52:52.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters' 2024-09-13T16:52:52.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.651+0000 7f2cf7000700 1 Processor -- start 2024-09-13T16:52:52.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.651+0000 7f2cf7000700 1 -- start start 2024-09-13T16:52:52.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cf7000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2cf00b5c50 0x7f2cf00cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cf7000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf015c0e0 con 0x7f2cf006fa00 2024-09-13T16:52:52.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cf7000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf00b4000 con 0x7f2cf0155250 2024-09-13T16:52:52.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cf7000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf0155630 con 0x7f2cf00b5c50 2024-09-13T16:52:52.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cf5000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2cf006fa00 0x7f2cf00b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58800/0 (socket says 172.21.15.3:58800) 2024-09-13T16:52:52.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cf5000700 1 -- 172.21.15.3:0/95541514 learned_addr learned my addr 172.21.15.3:0/95541514 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:52.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cef400700 1 --2- 172.21.15.3:0/95541514 >> v2:172.21.15.157:3300/0 conn(0x7f2cf00b5c50 0x7f2cf00cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2ceea00700 1 -- 172.21.15.3:0/95541514 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300853991 0 0) 0x7f2cf015c0e0 con 0x7f2cf006fa00 2024-09-13T16:52:52.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2ceea00700 1 -- 172.21.15.3:0/95541514 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd40035d0 con 0x7f2cf006fa00 2024-09-13T16:52:52.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cef400700 1 -- 172.21.15.3:0/95541514 >> v1:172.21.15.157:6789/0 conn(0x7f2cf0155250 legacy=0x7f2cf015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2ceea00700 1 -- 172.21.15.3:0/95541514 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510457821 0 0) 0x7f2cf00b4000 con 0x7f2cf0155250 2024-09-13T16:52:52.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cef400700 1 -- 172.21.15.3:0/95541514 >> v1:172.21.15.3:6789/0 conn(0x7f2cf006fa00 legacy=0x7f2cf00b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cef400700 1 -- 172.21.15.3:0/95541514 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00b60c0 con 0x7f2cf00b5c50 2024-09-13T16:52:52.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2cef400700 1 --2- 172.21.15.3:0/95541514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 0x7f2cf00cf360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009850 tx=0x7f2cdc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d63a810ddcf974d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2ceea00700 1 -- 172.21.15.3:0/95541514 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc011830 con 0x7f2cf00b5c50 2024-09-13T16:52:52.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2ceea00700 1 -- 172.21.15.3:0/95541514 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc011990 con 0x7f2cf00b5c50 2024-09-13T16:52:52.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.655+0000 7f2ceea00700 1 -- 172.21.15.3:0/95541514 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc011d00 con 0x7f2cf00b5c50 2024-09-13T16:52:52.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 -- 172.21.15.3:0/95541514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 msgr2=0x7f2cf00cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 --2- 172.21.15.3:0/95541514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 0x7f2cf00cf360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009850 tx=0x7f2cdc009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:52.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 -- 172.21.15.3:0/95541514 shutdown_connections 2024-09-13T16:52:52.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 --2- 172.21.15.3:0/95541514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 0x7f2cf00cf360 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 -- 172.21.15.3:0/95541514 >> 172.21.15.3:0/95541514 conn(0x7f2cf00b2ad0 msgr2=0x7f2cf00b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:52.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 -- 172.21.15.3:0/95541514 shutdown_connections 2024-09-13T16:52:52.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 -- 172.21.15.3:0/95541514 wait complete. 2024-09-13T16:52:52.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 Processor -- start 2024-09-13T16:52:52.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.659+0000 7f2cf7000700 1 -- start start 2024-09-13T16:52:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf7000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf006fa00 0x7f2cf013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf7000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 0x7f2cf013e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf7000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cf0155250 0x7f2cf015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf7000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0156700 con 0x7f2cf0155250 2024-09-13T16:52:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf7000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf013df40 con 0x7f2cf00b5c50 2024-09-13T16:52:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf7000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0156f10 con 0x7f2cf006fa00 2024-09-13T16:52:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cf0155250 0x7f2cf015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cf0155250 0x7f2cf015a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35280/0 (socket says 172.21.15.3:35280) 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 -- 172.21.15.3:0/3679425382 learned_addr learned my addr 172.21.15.3:0/3679425382 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cef400700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 0x7f2cf013e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cefe00700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf006fa00 0x7f2cf013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 -- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf006fa00 msgr2=0x7f2cf013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf006fa00 0x7f2cf013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 -- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 msgr2=0x7f2cf013e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 0x7f2cf013e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 -- 172.21.15.3:0/3679425382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0094b0 con 0x7f2cf0155250 2024-09-13T16:52:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf5000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cf0155250 0x7f2cf015a370 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800cad0 tx=0x7f2ce800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2ce7e00700 1 -- 172.21.15.3:0/3679425382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8007500 con 0x7f2cf0155250 2024-09-13T16:52:52.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2ce7e00700 1 -- 172.21.15.3:0/3679425382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce8007660 con 0x7f2cf0155250 2024-09-13T16:52:52.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2ce7e00700 1 -- 172.21.15.3:0/3679425382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80079f0 con 0x7f2cf0155250 2024-09-13T16:52:52.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.663+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf01571f0 con 0x7f2cf0155250 2024-09-13T16:52:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.667+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf01577b0 con 0x7f2cf0155250 2024-09-13T16:52:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.667+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf01579a0 con 0x7f2cf0155250 2024-09-13T16:52:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.667+0000 7f2ce7e00700 1 -- 172.21.15.3:0/3679425382 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ce801b3b0 con 0x7f2cf0155250 2024-09-13T16:52:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.671+0000 7f2ce7e00700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2cd0066730 0x7f2cd0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.671+0000 7f2ce7e00700 1 -- 172.21.15.3:0/3679425382 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f2ce8083a30 con 0x7f2cf0155250 2024-09-13T16:52:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.671+0000 7f2cefe00700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2cd0066730 0x7f2cd0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.671+0000 7f2cefe00700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2cd0066730 0x7f2cd0068bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0000f60 tx=0x7f2ce0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.675+0000 7f2ce7e00700 1 -- 172.21.15.3:0/3679425382 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2ce8050ef0 con 0x7f2cf0155250 2024-09-13T16:52:52.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:52.927+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]} v 0) v1 -- 0x7f2cf00cf360 con 0x7f2cf0155250 2024-09-13T16:52:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.543+0000 7f2ce7e00700 1 -- 172.21.15.3:0/3679425382 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]}]=0 v146) v1 ==== 146+0+0 (secure 0 0 0) 0x7f2ce8017020 con 0x7f2cf0155250 2024-09-13T16:52:53.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2cd0066730 msgr2=0x7f2cd0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:53.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2cd0066730 0x7f2cd0068bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0000f60 tx=0x7f2ce0009040 comp rx=0 tx=0).stop 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cf0155250 msgr2=0x7f2cf015a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cf0155250 0x7f2cf015a370 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800cad0 tx=0x7f2ce800cb00 comp rx=0 tx=0).stop 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 shutdown_connections 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2cd0066730 0x7f2cd0068bf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cf0155250 0x7f2cf015a370 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf00b5c50 0x7f2cf013e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 --2- 172.21.15.3:0/3679425382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf006fa00 0x7f2cf013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 >> 172.21.15.3:0/3679425382 conn(0x7f2cf00b2ad0 msgr2=0x7f2cf0154570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 shutdown_connections 2024-09-13T16:52:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.551+0000 7f2cf7000700 1 -- 172.21.15.3:0/3679425382 wait complete. 2024-09-13T16:52:53.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T16:52:53.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.759+0000 7fb127200700 1 Processor -- start 2024-09-13T16:52:53.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.759+0000 7fb127200700 1 -- start start 2024-09-13T16:52:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb127200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1201507b0 0x7fb120159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb127200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb1200b2ce0 0x7fb12006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb127200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb12006c650 0x7fb12006a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb127200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12006e180 con 0x7fb1200b2ce0 2024-09-13T16:52:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb127200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12006cba0 con 0x7fb12006c650 2024-09-13T16:52:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb127200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12006ce80 con 0x7fb1201507b0 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb1200b2ce0 0x7fb12006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb1200b2ce0 0x7fb12006dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39690/0 (socket says 172.21.15.3:39690) 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 -- 172.21.15.3:0/3612029492 learned_addr learned my addr 172.21.15.3:0/3612029492 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125e00700 1 --2- 172.21.15.3:0/3612029492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1201507b0 0x7fb120159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb126800700 1 --2- 172.21.15.3:0/3612029492 >> v2:172.21.15.157:3300/0 conn(0x7fb12006c650 0x7fb12006a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 -- 172.21.15.3:0/3612029492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1201507b0 msgr2=0x7fb120159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 --2- 172.21.15.3:0/3612029492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1201507b0 0x7fb120159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 -- 172.21.15.3:0/3612029492 >> v2:172.21.15.157:3300/0 conn(0x7fb12006c650 msgr2=0x7fb12006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 --2- 172.21.15.3:0/3612029492 >> v2:172.21.15.157:3300/0 conn(0x7fb12006c650 0x7fb12006a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 -- 172.21.15.3:0/3612029492 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12006d160 con 0x7fb1200b2ce0 2024-09-13T16:52:53.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb126800700 1 --2- 172.21.15.3:0/3612029492 >> v2:172.21.15.157:3300/0 conn(0x7fb12006c650 0x7fb12006a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:53.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb125400700 1 --2- 172.21.15.3:0/3612029492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb12006dc40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb1200b4770 tx=0x7fb110009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85f983bb68c8640 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:53.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb124a00700 1 -- 172.21.15.3:0/3612029492 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000b8a0 con 0x7fb1200b2ce0 2024-09-13T16:52:53.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb124a00700 1 -- 172.21.15.3:0/3612029492 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11000ba00 con 0x7fb1200b2ce0 2024-09-13T16:52:53.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.763+0000 7fb124a00700 1 -- 172.21.15.3:0/3612029492 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000bd90 con 0x7fb1200b2ce0 2024-09-13T16:52:53.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 -- 172.21.15.3:0/3612029492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 msgr2=0x7fb12006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:53.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 --2- 172.21.15.3:0/3612029492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb12006dc40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb1200b4770 tx=0x7fb110009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 -- 172.21.15.3:0/3612029492 shutdown_connections 2024-09-13T16:52:53.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 --2- 172.21.15.3:0/3612029492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb12006dc40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 --2- 172.21.15.3:0/3612029492 >> v2:172.21.15.157:3300/0 conn(0x7fb12006c650 0x7fb12006a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 --2- 172.21.15.3:0/3612029492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1201507b0 0x7fb120159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 -- 172.21.15.3:0/3612029492 >> 172.21.15.3:0/3612029492 conn(0x7fb1200af620 msgr2=0x7fb1200afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:53.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 -- 172.21.15.3:0/3612029492 shutdown_connections 2024-09-13T16:52:53.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 -- 172.21.15.3:0/3612029492 wait complete. 2024-09-13T16:52:53.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.767+0000 7fb127200700 1 Processor -- start 2024-09-13T16:52:53.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb127200700 1 -- start start 2024-09-13T16:52:53.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb127200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12006c650 0x7fb120137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:53.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb127200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb1201e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:53.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb127200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1201507b0 0x7fb1201f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:53.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb127200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb120148950 con 0x7fb1200b2ce0 2024-09-13T16:52:53.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb127200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb120147d80 con 0x7fb1201507b0 2024-09-13T16:52:53.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb127200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb120148020 con 0x7fb12006c650 2024-09-13T16:52:53.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb1201e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:53.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb1201e1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39692/0 (socket says 172.21.15.3:39692) 2024-09-13T16:52:53.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 -- 172.21.15.3:0/1126503301 learned_addr learned my addr 172.21.15.3:0/1126503301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:53.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb126800700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1201507b0 0x7fb1201f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:53.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125e00700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12006c650 0x7fb120137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:53.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 -- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12006c650 msgr2=0x7fb120137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:53.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12006c650 0x7fb120137c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 -- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1201507b0 msgr2=0x7fb1201f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:53.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1201507b0 0x7fb1201f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 -- 172.21.15.3:0/1126503301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb110009570 con 0x7fb1200b2ce0 2024-09-13T16:52:53.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb125400700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb1201e1e40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb1200b1620 tx=0x7fb110004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.771+0000 7fb11a000700 1 -- 172.21.15.3:0/1126503301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100036c0 con 0x7fb1200b2ce0 2024-09-13T16:52:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.775+0000 7fb11a000700 1 -- 172.21.15.3:0/1126503301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11001f070 con 0x7fb1200b2ce0 2024-09-13T16:52:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.775+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb120148300 con 0x7fb1200b2ce0 2024-09-13T16:52:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.775+0000 7fb11a000700 1 -- 172.21.15.3:0/1126503301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110003a80 con 0x7fb1200b2ce0 2024-09-13T16:52:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.775+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1201e2380 con 0x7fb1200b2ce0 2024-09-13T16:52:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.775+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12006dc40 con 0x7fb1200b2ce0 2024-09-13T16:52:53.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.775+0000 7fb11a000700 1 -- 172.21.15.3:0/1126503301 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb110021030 con 0x7fb1200b2ce0 2024-09-13T16:52:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.779+0000 7fb11a000700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb104066730 0x7fb104068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.779+0000 7fb11a000700 1 -- 172.21.15.3:0/1126503301 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fb1100856a0 con 0x7fb1200b2ce0 2024-09-13T16:52:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.779+0000 7fb125e00700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb104066730 0x7fb104068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.779+0000 7fb125e00700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb104066730 0x7fb104068bf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb11400b9d0 tx=0x7fb114008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.783+0000 7fb11a000700 1 -- 172.21.15.3:0/1126503301 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb110052b60 con 0x7fb1200b2ce0 2024-09-13T16:52:53.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.971+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb1200b4a60 con 0x7fb1200b2ce0 2024-09-13T16:52:53.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.975+0000 7fb11a000700 1 -- 172.21.15.3:0/1126503301 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb110056180 con 0x7fb1200b2ce0 2024-09-13T16:52:53.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb104066730 msgr2=0x7fb104068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:53.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb104066730 0x7fb104068bf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb11400b9d0 tx=0x7fb114008040 comp rx=0 tx=0).stop 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 msgr2=0x7fb1201e1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb1201e1e40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb1200b1620 tx=0x7fb110004770 comp rx=0 tx=0).stop 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 shutdown_connections 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb104066730 0x7fb104068bf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1201507b0 0x7fb1201f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1200b2ce0 0x7fb1201e1e40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 --2- 172.21.15.3:0/1126503301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12006c650 0x7fb120137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 >> 172.21.15.3:0/1126503301 conn(0x7fb1200af620 msgr2=0x7fb120071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 shutdown_connections 2024-09-13T16:52:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:53.979+0000 7fb127200700 1 -- 172.21.15.3:0/1126503301 wait complete. 2024-09-13T16:52:54.008 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T16:52:54.008 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T16:52:54.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T16:52:54.092 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T16:52:54.092 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:54.093 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:54.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349932k,nr_inodes=4087483,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4619 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T16:52:54.113 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:52:54.114 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:54.155 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346844k,nr_inodes=4086711,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T16:52:54.155 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:52:54.155 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:54.178 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349932k,nr_inodes=4087483,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4619 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T16:52:54.179 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:52:54.179 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:52:54.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346844k,nr_inodes=4086711,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T16:52:54.214 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T16:52:54.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.387+0000 7f0dc4800700 1 Processor -- start 2024-09-13T16:52:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc4800700 1 -- start start 2024-09-13T16:52:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc4800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dbc0cc420 0x7f0dbc15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc4800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc4800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc06d4c0 con 0x7f0dbc0cc800 2024-09-13T16:52:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc4800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc15c160 con 0x7f0dbc0cc420 2024-09-13T16:52:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc4800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc071320 con 0x7f0dbc0ce520 2024-09-13T16:52:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc1400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc1e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dbc0cc420 0x7f0dbc15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc1400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:55028/0 (socket says 172.21.15.3:55028) 2024-09-13T16:52:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc1400700 1 -- 172.21.15.3:0/4291998015 learned_addr learned my addr 172.21.15.3:0/4291998015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.391+0000 7f0dc1e00700 1 --2- 172.21.15.3:0/4291998015 >> v2:172.21.15.157:3300/0 conn(0x7f0dbc0cc420 0x7f0dbc15bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:57646/0 (socket says 172.21.15.3:57646) 2024-09-13T16:52:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc1e00700 1 -- 172.21.15.3:0/4291998015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 msgr2=0x7f0dbc158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc1e00700 1 --2- 172.21.15.3:0/4291998015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc1e00700 1 -- 172.21.15.3:0/4291998015 >> v1:172.21.15.157:6789/0 conn(0x7f0dbc0cc800 legacy=0x7f0dbc15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc1e00700 1 -- 172.21.15.3:0/4291998015 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc071600 con 0x7f0dbc0cc420 2024-09-13T16:52:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc1400700 1 --2- 172.21.15.3:0/4291998015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc1e00700 1 --2- 172.21.15.3:0/4291998015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 0x7f0dbc15bc20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009850 tx=0x7f0db0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b45d8a6601a5f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc0a00700 1 -- 172.21.15.3:0/4291998015 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0011830 con 0x7f0dbc0cc420 2024-09-13T16:52:54.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc0a00700 1 -- 172.21.15.3:0/4291998015 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db0011990 con 0x7f0dbc0cc420 2024-09-13T16:52:54.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc0a00700 1 -- 172.21.15.3:0/4291998015 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0011d40 con 0x7f0dbc0cc420 2024-09-13T16:52:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc4800700 1 -- 172.21.15.3:0/4291998015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 msgr2=0x7f0dbc15bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc4800700 1 --2- 172.21.15.3:0/4291998015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 0x7f0dbc15bc20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009850 tx=0x7f0db0009b60 comp rx=0 tx=0).stop 2024-09-13T16:52:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc4800700 1 -- 172.21.15.3:0/4291998015 shutdown_connections 2024-09-13T16:52:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc4800700 1 --2- 172.21.15.3:0/4291998015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc4800700 1 --2- 172.21.15.3:0/4291998015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 0x7f0dbc15bc20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc4800700 1 -- 172.21.15.3:0/4291998015 >> 172.21.15.3:0/4291998015 conn(0x7f0dbc06a390 msgr2=0x7f0dbc06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc4800700 1 -- 172.21.15.3:0/4291998015 shutdown_connections 2024-09-13T16:52:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.395+0000 7f0dc4800700 1 -- 172.21.15.3:0/4291998015 wait complete. 2024-09-13T16:52:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.399+0000 7f0dc4800700 1 Processor -- start 2024-09-13T16:52:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.399+0000 7f0dc4800700 1 -- start start 2024-09-13T16:52:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc4800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 0x7f0dbc148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc4800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc0cc800 0x7f0dbc14b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc4800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc4800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc148ea0 con 0x7f0dbc0cc800 2024-09-13T16:52:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc4800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc14a1c0 con 0x7f0dbc0cc420 2024-09-13T16:52:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc4800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc14a4a0 con 0x7f0dbc0ce520 2024-09-13T16:52:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc0cc800 0x7f0dbc14b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc0cc800 0x7f0dbc14b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39704/0 (socket says 172.21.15.3:39704) 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 -- 172.21.15.3:0/3592438452 learned_addr learned my addr 172.21.15.3:0/3592438452 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1e00700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 0x7f0dbc148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc2800700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 -- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 msgr2=0x7f0dbc158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 -- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 msgr2=0x7f0dbc148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 0x7f0dbc148960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 -- 172.21.15.3:0/3592438452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac009570 con 0x7f0dbc0cc800 2024-09-13T16:52:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc1400700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc0cc800 0x7f0dbc14b260 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00e9d0 tx=0x7f0dac00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0db6000700 1 -- 172.21.15.3:0/3592438452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac00ee70 con 0x7f0dbc0cc800 2024-09-13T16:52:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0db6000700 1 -- 172.21.15.3:0/3592438452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac00c850 con 0x7f0dbc0cc800 2024-09-13T16:52:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0db6000700 1 -- 172.21.15.3:0/3592438452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac00cbc0 con 0x7f0dbc0cc800 2024-09-13T16:52:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db00094b0 con 0x7f0dbc0cc800 2024-09-13T16:52:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.403+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc14aa60 con 0x7f0dbc0cc800 2024-09-13T16:52:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.407+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc15bc20 con 0x7f0dbc0cc800 2024-09-13T16:52:54.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.407+0000 7f0db6000700 1 -- 172.21.15.3:0/3592438452 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0dac00cd20 con 0x7f0dbc0cc800 2024-09-13T16:52:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.411+0000 7f0db6000700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.411+0000 7f0db6000700 1 -- 172.21.15.3:0/3592438452 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f0dac083cd0 con 0x7f0dbc0cc800 2024-09-13T16:52:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.411+0000 7f0dc1e00700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.411+0000 7f0dc1e00700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0da0066730 0x7f0da0068bf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0db00090b0 tx=0x7f0db0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.415+0000 7f0db6000700 1 -- 172.21.15.3:0/3592438452 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0dac051100 con 0x7f0dbc0cc800 2024-09-13T16:52:54.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.651+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dbc0706f0 con 0x7f0dbc0cc800 2024-09-13T16:52:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.651+0000 7f0db6000700 1 -- 172.21.15.3:0/3592438452 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f0dac021de0 con 0x7f0dbc0cc800 2024-09-13T16:52:54.660 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:54.660 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4119860563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4119860563},{"type":"v1","addr":"172.21.15.157:6835","nonce":4119860563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2247712482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2247712482},{"type":"v1","addr":"172.21.15.3:6834","nonce":2247712482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1957501779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1957501779},{"type":"v1","addr":"172.21.15.157:6837","nonce":1957501779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1459884734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1459884734},{"type":"v1","addr":"172.21.15.3:6835","nonce":1459884734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[]} 2024-09-13T16:52:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0da0066730 msgr2=0x7f0da0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0da0066730 0x7f0da0068bf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0db00090b0 tx=0x7f0db0008030 comp rx=0 tx=0).stop 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc0cc800 msgr2=0x7f0dbc14b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc0cc800 0x7f0dbc14b260 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00e9d0 tx=0x7f0dac00ece0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 shutdown_connections 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0ce520 0x7f0dbc158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc0cc800 0x7f0dbc14b260 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 --2- 172.21.15.3:0/3592438452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc0cc420 0x7f0dbc148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 >> 172.21.15.3:0/3592438452 conn(0x7f0dbc06a390 msgr2=0x7f0dbc15e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 shutdown_connections 2024-09-13T16:52:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.659+0000 7f0dc4800700 1 -- 172.21.15.3:0/3592438452 wait complete. 2024-09-13T16:52:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 24 2024-09-13T16:52:54.689 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:52:54.689 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T16:52:54.689 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:52:54.690 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:54.690 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T16:52:54.690 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T16:52:54.690 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T16:52:54.690 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:54.690 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:52:54.690 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T16:52:54.691 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:52:54.691 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:54.691 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:52:54.691 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T16:52:54.691 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:52:54.691 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:54.714 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:52:54.714 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:52:54.715 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:52:54.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:52:54.720 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T16:52:54.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T16:52:54.721 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:52:54.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:52:54.766 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:52:54.766 DEBUG:teuthology.parallel:result is None 2024-09-13T16:52:54.769 INFO:tasks.ceph.mds.c:Started 2024-09-13T16:52:54.769 DEBUG:teuthology.parallel:result is None 2024-09-13T16:52:54.770 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:52:54.770 DEBUG:teuthology.parallel:result is None 2024-09-13T16:52:54.771 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:52:54.771 DEBUG:teuthology.parallel:result is None 2024-09-13T16:52:54.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T16:52:54.884 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-13T16:52:54.886 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-13T16:52:54.889 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-13T16:52:54.892 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-13T16:52:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf82800700 1 Processor -- start 2024-09-13T16:52:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf82800700 1 -- start start 2024-09-13T16:52:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf82800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf7c06ccf0 0x7fcf7c165820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf82800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c0b4100 con 0x7fcf7c144790 2024-09-13T16:52:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf82800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c0ce8e0 con 0x7fcf7c1443b0 2024-09-13T16:52:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf82800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c0ccd30 con 0x7fcf7c06ccf0 2024-09-13T16:52:54.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf81e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf7c06ccf0 0x7fcf7c165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf80a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcf7c144790 0x7fcf7c06c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44294/0 (socket says 172.21.15.3:44294) 2024-09-13T16:52:54.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.931+0000 7fcf80a00700 1 -- 172.21.15.3:0/220760944 learned_addr learned my addr 172.21.15.3:0/220760944 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf81e00700 1 -- 172.21.15.3:0/220760944 >> v1:172.21.15.157:6789/0 conn(0x7fcf7c1443b0 legacy=0x7fcf7c0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf81e00700 1 -- 172.21.15.3:0/220760944 >> v1:172.21.15.3:6789/0 conn(0x7fcf7c144790 legacy=0x7fcf7c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf7be00700 1 -- 172.21.15.3:0/220760944 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407945806 0 0) 0x7fcf7c0b4100 con 0x7fcf7c144790 2024-09-13T16:52:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf81e00700 1 -- 172.21.15.3:0/220760944 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c0cd010 con 0x7fcf7c06ccf0 2024-09-13T16:52:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf81e00700 1 --2- 172.21.15.3:0/220760944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c06ccf0 0x7fcf7c165820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009d90 tx=0x7fcf7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e19f1d5ba67e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf7be00700 1 -- 172.21.15.3:0/220760944 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400e070 con 0x7fcf7c06ccf0 2024-09-13T16:52:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf7be00700 1 -- 172.21.15.3:0/220760944 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7400bbd0 con 0x7fcf7c06ccf0 2024-09-13T16:52:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf7be00700 1 -- 172.21.15.3:0/220760944 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74012430 con 0x7fcf7c06ccf0 2024-09-13T16:52:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf82800700 1 -- 172.21.15.3:0/220760944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c06ccf0 msgr2=0x7fcf7c165820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf82800700 1 --2- 172.21.15.3:0/220760944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c06ccf0 0x7fcf7c165820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009d90 tx=0x7fcf7400b330 comp rx=0 tx=0).stop 2024-09-13T16:52:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf82800700 1 -- 172.21.15.3:0/220760944 shutdown_connections 2024-09-13T16:52:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf82800700 1 --2- 172.21.15.3:0/220760944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c06ccf0 0x7fcf7c165820 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf82800700 1 -- 172.21.15.3:0/220760944 >> 172.21.15.3:0/220760944 conn(0x7fcf7c0b0cf0 msgr2=0x7fcf7c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf82800700 1 -- 172.21.15.3:0/220760944 shutdown_connections 2024-09-13T16:52:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.935+0000 7fcf82800700 1 -- 172.21.15.3:0/220760944 wait complete. 2024-09-13T16:52:54.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.939+0000 7fcf82800700 1 Processor -- start 2024-09-13T16:52:54.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.939+0000 7fcf82800700 1 -- start start 2024-09-13T16:52:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf82800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c06ccf0 0x7fcf7c136bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf82800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c1443b0 0x7fcf7c156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf82800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c1571a0 0x7fcf7c13f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf82800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c157c70 con 0x7fcf7c1443b0 2024-09-13T16:52:54.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf82800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1576b0 con 0x7fcf7c1571a0 2024-09-13T16:52:54.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf82800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c157990 con 0x7fcf7c06ccf0 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c1443b0 0x7fcf7c156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c1443b0 0x7fcf7c156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39740/0 (socket says 172.21.15.3:39740) 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 -- 172.21.15.3:0/3094895625 learned_addr learned my addr 172.21.15.3:0/3094895625 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf81400700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c06ccf0 0x7fcf7c136bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf81e00700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c1571a0 0x7fcf7c13f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 -- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c06ccf0 msgr2=0x7fcf7c136bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c06ccf0 0x7fcf7c136bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 -- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c1571a0 msgr2=0x7fcf7c13f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c1571a0 0x7fcf7c13f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 -- 172.21.15.3:0/3094895625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74009a70 con 0x7fcf7c1443b0 2024-09-13T16:52:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf80a00700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c1443b0 0x7fcf7c156c60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c4e0 tx=0x7fcf6800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf79600700 1 -- 172.21.15.3:0/3094895625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68004ca0 con 0x7fcf7c1443b0 2024-09-13T16:52:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf79600700 1 -- 172.21.15.3:0/3094895625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf68004e20 con 0x7fcf7c1443b0 2024-09-13T16:52:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf79600700 1 -- 172.21.15.3:0/3094895625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68007310 con 0x7fcf7c1443b0 2024-09-13T16:52:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c134cb0 con 0x7fcf7c1443b0 2024-09-13T16:52:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.943+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c137660 con 0x7fcf7c1443b0 2024-09-13T16:52:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.947+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c06c5e0 con 0x7fcf7c1443b0 2024-09-13T16:52:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.947+0000 7fcf79600700 1 -- 172.21.15.3:0/3094895625 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf68007470 con 0x7fcf7c1443b0 2024-09-13T16:52:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.951+0000 7fcf79600700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf5c066730 0x7fcf5c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.951+0000 7fcf79600700 1 -- 172.21.15.3:0/3094895625 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fcf68083c40 con 0x7fcf7c1443b0 2024-09-13T16:52:54.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.951+0000 7fcf81400700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf5c066730 0x7fcf5c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.951+0000 7fcf81400700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf5c066730 0x7fcf5c068bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcf70001740 tx=0x7fcf70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:54.959+0000 7fcf79600700 1 -- 172.21.15.3:0/3094895625 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf68051100 con 0x7fcf7c1443b0 2024-09-13T16:52:55.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.159+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcf7c0afbb0 con 0x7fcf7c1443b0 2024-09-13T16:52:55.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf79600700 1 -- 172.21.15.3:0/3094895625 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcf68007720 con 0x7fcf7c1443b0 2024-09-13T16:52:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf5c066730 msgr2=0x7fcf5c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf5c066730 0x7fcf5c068bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcf70001740 tx=0x7fcf70009040 comp rx=0 tx=0).stop 2024-09-13T16:52:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c1443b0 msgr2=0x7fcf7c156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c1443b0 0x7fcf7c156c60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c4e0 tx=0x7fcf6800c7f0 comp rx=0 tx=0).stop 2024-09-13T16:52:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 shutdown_connections 2024-09-13T16:52:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf5c066730 0x7fcf5c068bf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c1571a0 0x7fcf7c13f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c1443b0 0x7fcf7c156c60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 --2- 172.21.15.3:0/3094895625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c06ccf0 0x7fcf7c136bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.927+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 >> 172.21.15.3:0/3094895625 conn(0x7fcf7c0b0cf0 msgr2=0x7fcf7c0b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.931+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 shutdown_connections 2024-09-13T16:52:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:55.931+0000 7fcf82800700 1 -- 172.21.15.3:0/3094895625 wait complete. 2024-09-13T16:52:55.938 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-13T16:52:55.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T16:52:56.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.139+0000 7f4a0b200700 1 Processor -- start 2024-09-13T16:52:56.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.139+0000 7f4a0b200700 1 -- start start 2024-09-13T16:52:56.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a0b200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a040ceaa0 0x7f4a0415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a0b200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0415aa00 con 0x7f4a040b47a0 2024-09-13T16:52:56.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a0b200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0415ace0 con 0x7f4a040b4b80 2024-09-13T16:52:56.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a0b200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040cdf70 con 0x7f4a040ceaa0 2024-09-13T16:52:56.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a09e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4a040b47a0 0x7f4a0406d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44310/0 (socket says 172.21.15.3:44310) 2024-09-13T16:52:56.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a09e00700 1 -- 172.21.15.3:0/3419805576 learned_addr learned my addr 172.21.15.3:0/3419805576 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:56.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a0a800700 1 --2- 172.21.15.3:0/3419805576 >> v2:172.21.15.3:3300/0 conn(0x7f4a040ceaa0 0x7f4a0415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a08a00700 1 -- 172.21.15.3:0/3419805576 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823680835 0 0) 0x7f4a0415aa00 con 0x7f4a040b47a0 2024-09-13T16:52:56.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.143+0000 7f4a08a00700 1 -- 172.21.15.3:0/3419805576 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ec0035d0 con 0x7f4a040b47a0 2024-09-13T16:52:56.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a08a00700 1 -- 172.21.15.3:0/3419805576 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154045734 0 0) 0x7f4a0415ace0 con 0x7f4a040b4b80 2024-09-13T16:52:56.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a08a00700 1 -- 172.21.15.3:0/3419805576 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a0415aa00 con 0x7f4a040b4b80 2024-09-13T16:52:56.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a08a00700 1 -- 172.21.15.3:0/3419805576 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562544041 0 0) 0x7f49ec0035d0 con 0x7f4a040b47a0 2024-09-13T16:52:56.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0a800700 1 -- 172.21.15.3:0/3419805576 >> v1:172.21.15.157:6789/0 conn(0x7f4a040b4b80 legacy=0x7f4a040ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0a800700 1 -- 172.21.15.3:0/3419805576 >> v1:172.21.15.3:6789/0 conn(0x7f4a040b47a0 legacy=0x7f4a0406d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0a800700 1 -- 172.21.15.3:0/3419805576 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040cdb90 con 0x7f4a040ceaa0 2024-09-13T16:52:56.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0a800700 1 --2- 172.21.15.3:0/3419805576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a0415a3f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d90 tx=0x7f4a0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f178878a7a82c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:56.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a08a00700 1 -- 172.21.15.3:0/3419805576 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0000e070 con 0x7f4a040ceaa0 2024-09-13T16:52:56.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a08a00700 1 -- 172.21.15.3:0/3419805576 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a00003a40 con 0x7f4a040ceaa0 2024-09-13T16:52:56.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a08a00700 1 -- 172.21.15.3:0/3419805576 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8001670 con 0x7f4a040ceaa0 2024-09-13T16:52:56.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0b200700 1 -- 172.21.15.3:0/3419805576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 msgr2=0x7f4a0415a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0b200700 1 --2- 172.21.15.3:0/3419805576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a0415a3f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d90 tx=0x7f4a0000b330 comp rx=0 tx=0).stop 2024-09-13T16:52:56.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0b200700 1 -- 172.21.15.3:0/3419805576 shutdown_connections 2024-09-13T16:52:56.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0b200700 1 --2- 172.21.15.3:0/3419805576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a0415a3f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0b200700 1 -- 172.21.15.3:0/3419805576 >> 172.21.15.3:0/3419805576 conn(0x7f4a04071da0 msgr2=0x7f4a040b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:56.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0b200700 1 -- 172.21.15.3:0/3419805576 shutdown_connections 2024-09-13T16:52:56.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.147+0000 7f4a0b200700 1 -- 172.21.15.3:0/3419805576 wait complete. 2024-09-13T16:52:56.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0b200700 1 Processor -- start 2024-09-13T16:52:56.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0b200700 1 -- start start 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a040b47a0 0x7f4a04153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0b200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a040b4b80 0x7f4a04151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0b200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a041523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0b200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04150ce0 con 0x7f4a040ceaa0 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a041562f0 con 0x7f4a040b47a0 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0b200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a041565d0 con 0x7f4a040b4b80 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a041523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a041523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39760/0 (socket says 172.21.15.3:39760) 2024-09-13T16:52:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 -- 172.21.15.3:0/3932118207 learned_addr learned my addr 172.21.15.3:0/3932118207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a09e00700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a040b47a0 0x7f4a04153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a09400700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a040b4b80 0x7f4a04151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 -- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a040b4b80 msgr2=0x7f4a04151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a040b4b80 0x7f4a04151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 -- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a040b47a0 msgr2=0x7f4a04153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a040b47a0 0x7f4a04153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 -- 172.21.15.3:0/3932118207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00009a70 con 0x7f4a040ceaa0 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.151+0000 7f4a0a800700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a041523f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000b370 tx=0x7f4a00004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:56.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.155+0000 7f49fe000700 1 -- 172.21.15.3:0/3932118207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000045c0 con 0x7f4a040ceaa0 2024-09-13T16:52:56.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.155+0000 7f49fe000700 1 -- 172.21.15.3:0/3932118207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a00004720 con 0x7f4a040ceaa0 2024-09-13T16:52:56.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.155+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a041568b0 con 0x7f4a040ceaa0 2024-09-13T16:52:56.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.155+0000 7f49fe000700 1 -- 172.21.15.3:0/3932118207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00012430 con 0x7f4a040ceaa0 2024-09-13T16:52:56.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.155+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0414fc20 con 0x7f4a040ceaa0 2024-09-13T16:52:56.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.155+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04151130 con 0x7f4a040ceaa0 2024-09-13T16:52:56.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.155+0000 7f49fe000700 1 -- 172.21.15.3:0/3932118207 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a000125b0 con 0x7f4a040ceaa0 2024-09-13T16:52:56.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.159+0000 7f49fe000700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f49e8066760 0x7f49e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.159+0000 7f49fe000700 1 -- 172.21.15.3:0/3932118207 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4a000853c0 con 0x7f4a040ceaa0 2024-09-13T16:52:56.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.159+0000 7f4a09e00700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f49e8066760 0x7f49e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.159+0000 7f4a09e00700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f49e8066760 0x7f49e8068c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f49f8001380 tx=0x7f49f8000f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.167+0000 7f49fe000700 1 -- 172.21.15.3:0/3932118207 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4a04151130 con 0x7f4a040ceaa0 2024-09-13T16:52:56.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.423+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4a04151130 con 0x7f4a040ceaa0 2024-09-13T16:52:56.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.423+0000 7f49fe000700 1 -- 172.21.15.3:0/3932118207 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4a04151130 con 0x7f4a040ceaa0 2024-09-13T16:52:56.430 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRm7PYhFhAAjGhG4+MGhybgDuUm80b3GA==", 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmwv/tKRAAkP5eDK1/mzVXdvg14IIyBQ==", 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmv2QBIRAAFCBAGuVsMDrEEbFHboF5fA==", 2024-09-13T16:52:56.432 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmqBN/NBAAcrl9JO1UMHIeJVdkAmMzBQ==", 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:52:56.433 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAfbeRmtMg4JBAAhpD6J8l0s10MToeuFpzBnQ==", 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.434 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAibeRmxu+sMxAAdKOLbj/zALwuUdqEcMGKOA==", 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAmbeRmneXPCBAA+5aq4baFkLLf9YgyT1Ej7A==", 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:52:56.435 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQApbeRmEJ5PGBAAm4ESpLC8pDcykDWH61UX+Q==", 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-13T16:52:56.436 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAtbeRmJ1v/NRAAVnN6OX537lXr9C2U5OwVcg==", 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAxbeRm1IErChAAlZJr0hgRM7hMjV9IKbFn5Q==", 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:52:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA0beRmjcW0FxAA8jqbdFW19Enk37M00fT5Mg==", 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.438 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA3beRmEURxJhAAbjA9Q82Jn0eyhkHLV3DnXg==", 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-13T16:52:56.439 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRm7UDIOhAAQ85WuAGFt3D40/vdHCx/SQ==", 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAcbeRm869WCxAALRQBL9frcnCHq6I4dwzKOA==", 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-13T16:52:56.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAcbeRmpROABRAA7JM6b7+quVMQ4WAlZ5Z0Kg==", 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.441 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAcbeRmYTniEBAALAPqutUd7WDZ6lUN3Zk4iQ==", 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAabeRm2q6YKRAAmAaMZZqHOf6hMHtPpgmMPQ==", 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.442 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmEAD+DhAAwiQnsaTbFldMIXOfTHEdow==", 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAbbeRmpP++CBAAYq8DHAVQrKZUzQMWAjwOQA==", 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-13T16:52:56.444 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:52:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f49e8066760 msgr2=0x7f49e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f49e8066760 0x7f49e8068c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f49f8001380 tx=0x7f49f8000f10 comp rx=0 tx=0).stop 2024-09-13T16:52:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 msgr2=0x7f4a041523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a041523f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000b370 tx=0x7f4a00004af0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 shutdown_connections 2024-09-13T16:52:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f49e8066760 0x7f49e8068c20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a040ceaa0 0x7f4a041523f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a040b4b80 0x7f4a04151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 --2- 172.21.15.3:0/3932118207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a040b47a0 0x7f4a04153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 >> 172.21.15.3:0/3932118207 conn(0x7f4a04071da0 msgr2=0x7f4a040cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 shutdown_connections 2024-09-13T16:52:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.427+0000 7f4a0b200700 1 -- 172.21.15.3:0/3932118207 wait complete. 2024-09-13T16:52:56.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:52:56.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.623+0000 7fd78ca00700 1 Processor -- start 2024-09-13T16:52:56.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd78ca00700 1 -- start start 2024-09-13T16:52:56.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd78ca00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd78806ccf0 0x7fd7881472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd78ca00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd78806fea0 0x7fd788070310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd78ca00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd788070850 con 0x7fd788146ee0 2024-09-13T16:52:56.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd78ca00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd788070a80 con 0x7fd78806fea0 2024-09-13T16:52:56.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd78ca00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7881626e0 con 0x7fd78806ccf0 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd787200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd78806fea0 0x7fd788070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd787200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd78806fea0 0x7fd788070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39762/0 (socket says 172.21.15.3:39762) 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd787200700 1 -- 172.21.15.3:0/2160202690 learned_addr learned my addr 172.21.15.3:0/2160202690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd77fe00700 1 --2- 172.21.15.3:0/2160202690 >> v2:172.21.15.157:3300/0 conn(0x7fd78806ccf0 0x7fd7881472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd787200700 1 -- 172.21.15.3:0/2160202690 >> v2:172.21.15.157:3300/0 conn(0x7fd78806ccf0 msgr2=0x7fd7881472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd787200700 1 --2- 172.21.15.3:0/2160202690 >> v2:172.21.15.157:3300/0 conn(0x7fd78806ccf0 0x7fd7881472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd787200700 1 -- 172.21.15.3:0/2160202690 >> v1:172.21.15.157:6789/0 conn(0x7fd788146ee0 legacy=0x7fd78806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd787200700 1 -- 172.21.15.3:0/2160202690 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788165800 con 0x7fd78806fea0 2024-09-13T16:52:56.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.627+0000 7fd785e00700 1 -- 172.21.15.3:0/2160202690 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148904245 0 0) 0x7fd788070850 con 0x7fd788146ee0 2024-09-13T16:52:56.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd787200700 1 --2- 172.21.15.3:0/2160202690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788070310 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd78806b830 tx=0x7fd78000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6e4ee78a45a102 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:56.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd785e00700 1 -- 172.21.15.3:0/2160202690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000e070 con 0x7fd78806fea0 2024-09-13T16:52:56.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd785e00700 1 -- 172.21.15.3:0/2160202690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78000b910 con 0x7fd78806fea0 2024-09-13T16:52:56.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd785e00700 1 -- 172.21.15.3:0/2160202690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000bcc0 con 0x7fd78806fea0 2024-09-13T16:52:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd78ca00700 1 -- 172.21.15.3:0/2160202690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 msgr2=0x7fd788070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd78ca00700 1 --2- 172.21.15.3:0/2160202690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788070310 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd78806b830 tx=0x7fd78000b330 comp rx=0 tx=0).stop 2024-09-13T16:52:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd78ca00700 1 -- 172.21.15.3:0/2160202690 shutdown_connections 2024-09-13T16:52:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd78ca00700 1 --2- 172.21.15.3:0/2160202690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788070310 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd78ca00700 1 --2- 172.21.15.3:0/2160202690 >> v2:172.21.15.157:3300/0 conn(0x7fd78806ccf0 0x7fd7881472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd78ca00700 1 -- 172.21.15.3:0/2160202690 >> 172.21.15.3:0/2160202690 conn(0x7fd7880b0cf0 msgr2=0x7fd7880b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd78ca00700 1 -- 172.21.15.3:0/2160202690 shutdown_connections 2024-09-13T16:52:56.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.631+0000 7fd78ca00700 1 -- 172.21.15.3:0/2160202690 wait complete. 2024-09-13T16:52:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd78ca00700 1 Processor -- start 2024-09-13T16:52:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd78ca00700 1 -- start start 2024-09-13T16:52:56.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd78ca00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd78806ccf0 0x7fd7881f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd78ca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788161860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd78ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd788146ee0 0x7fd788154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd78ca00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7881579e0 con 0x7fd78806fea0 2024-09-13T16:52:56.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd78ca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7881f4830 con 0x7fd788146ee0 2024-09-13T16:52:56.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd78ca00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7881fa920 con 0x7fd78806ccf0 2024-09-13T16:52:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd77fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd786800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd78806ccf0 0x7fd7881f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd77fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788161860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39776/0 (socket says 172.21.15.3:39776) 2024-09-13T16:52:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd77fe00700 1 -- 172.21.15.3:0/864279104 learned_addr learned my addr 172.21.15.3:0/864279104 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd787200700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd788146ee0 0x7fd788154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd77fe00700 1 -- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd78806ccf0 msgr2=0x7fd7881f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd77fe00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd78806ccf0 0x7fd7881f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd77fe00700 1 -- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd788146ee0 msgr2=0x7fd788154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd77fe00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd788146ee0 0x7fd788154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd77fe00700 1 -- 172.21.15.3:0/864279104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780009b30 con 0x7fd78806fea0 2024-09-13T16:52:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.635+0000 7fd786800700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd78806ccf0 0x7fd7881f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.639+0000 7fd77fe00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788161860 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd77000e9d0 tx=0x7fd77000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.639+0000 7fd77ea00700 1 -- 172.21.15.3:0/864279104 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd770009720 con 0x7fd78806fea0 2024-09-13T16:52:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.639+0000 7fd77ea00700 1 -- 172.21.15.3:0/864279104 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77000ee70 con 0x7fd78806fea0 2024-09-13T16:52:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.639+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7881fac00 con 0x7fd78806fea0 2024-09-13T16:52:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.639+0000 7fd77ea00700 1 -- 172.21.15.3:0/864279104 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77000ca50 con 0x7fd78806fea0 2024-09-13T16:52:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.639+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7881f6f10 con 0x7fd78806fea0 2024-09-13T16:52:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.639+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7881472c0 con 0x7fd78806fea0 2024-09-13T16:52:56.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.643+0000 7fd77ea00700 1 -- 172.21.15.3:0/864279104 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd77000cbb0 con 0x7fd78806fea0 2024-09-13T16:52:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.643+0000 7fd77ea00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd764066760 0x7fd764068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.643+0000 7fd77ea00700 1 -- 172.21.15.3:0/864279104 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd770083ec0 con 0x7fd78806fea0 2024-09-13T16:52:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.643+0000 7fd786800700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd764066760 0x7fd764068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.647+0000 7fd786800700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd764066760 0x7fd764068c20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd778000f60 tx=0x7fd778009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.651+0000 7fd77ea00700 1 -- 172.21.15.3:0/864279104 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd770051430 con 0x7fd78806fea0 2024-09-13T16:52:56.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.847+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7880ae620 con 0x7fd78806fea0 2024-09-13T16:52:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.851+0000 7fd77ea00700 1 -- 172.21.15.3:0/864279104 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+13014 (secure 0 0 0) 0x7fd77000ce60 con 0x7fd78806fea0 2024-09-13T16:52:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:56.858 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:52:55.919833+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:52:56.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd764066760 msgr2=0x7fd764068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd764066760 0x7fd764068c20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd778000f60 tx=0x7fd778009040 comp rx=0 tx=0).stop 2024-09-13T16:52:56.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 msgr2=0x7fd788161860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:56.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788161860 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd77000e9d0 tx=0x7fd77000ece0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 shutdown_connections 2024-09-13T16:52:56.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd764066760 0x7fd764068c20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd788146ee0 0x7fd788154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd78806fea0 0x7fd788161860 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 --2- 172.21.15.3:0/864279104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd78806ccf0 0x7fd7881f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:56.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 >> 172.21.15.3:0/864279104 conn(0x7fd7880b0cf0 msgr2=0x7fd7880cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:56.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 shutdown_connections 2024-09-13T16:52:56.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:56.855+0000 7fd78ca00700 1 -- 172.21.15.3:0/864279104 wait complete. 2024-09-13T16:52:56.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:52:56.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.063+0000 7f5939e00700 1 Processor -- start 2024-09-13T16:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.063+0000 7f5939e00700 1 -- start start 2024-09-13T16:52:57.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5939e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59341442c0 0x7f593406d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5939e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f593406dcb0 0x7f59340cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5939e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59340b41e0 con 0x7f59340cdd00 2024-09-13T16:52:57.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5939e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593406a390 con 0x7f59341442c0 2024-09-13T16:52:57.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5939e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593406b1f0 con 0x7f593406dcb0 2024-09-13T16:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5938a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59341442c0 0x7f593406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5938a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59341442c0 0x7f593406d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39778/0 (socket says 172.21.15.3:39778) 2024-09-13T16:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5938a00700 1 -- 172.21.15.3:0/1454524214 learned_addr learned my addr 172.21.15.3:0/1454524214 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5933e00700 1 --2- 172.21.15.3:0/1454524214 >> v2:172.21.15.157:3300/0 conn(0x7f593406dcb0 0x7f59340cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5938a00700 1 -- 172.21.15.3:0/1454524214 >> v2:172.21.15.157:3300/0 conn(0x7f593406dcb0 msgr2=0x7f59340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5938a00700 1 --2- 172.21.15.3:0/1454524214 >> v2:172.21.15.157:3300/0 conn(0x7f593406dcb0 0x7f59340cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5938a00700 1 -- 172.21.15.3:0/1454524214 >> v1:172.21.15.157:6789/0 conn(0x7f59340cdd00 legacy=0x7f59340b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5938a00700 1 -- 172.21.15.3:0/1454524214 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340ce170 con 0x7f59341442c0 2024-09-13T16:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5938a00700 1 --2- 172.21.15.3:0/1454524214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59341442c0 0x7f593406d770 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f59340ce300 tx=0x7f5928009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164afa4dcfc29d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5933400700 1 -- 172.21.15.3:0/1454524214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928004230 con 0x7f59341442c0 2024-09-13T16:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5933400700 1 -- 172.21.15.3:0/1454524214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928004390 con 0x7f59341442c0 2024-09-13T16:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5933400700 1 -- 172.21.15.3:0/1454524214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280036c0 con 0x7f59341442c0 2024-09-13T16:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5939e00700 1 -- 172.21.15.3:0/1454524214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59341442c0 msgr2=0x7f593406d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.067+0000 7f5939e00700 1 --2- 172.21.15.3:0/1454524214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59341442c0 0x7f593406d770 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f59340ce300 tx=0x7f5928009bf0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.071+0000 7f5939e00700 1 -- 172.21.15.3:0/1454524214 shutdown_connections 2024-09-13T16:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.071+0000 7f5939e00700 1 --2- 172.21.15.3:0/1454524214 >> v2:172.21.15.157:3300/0 conn(0x7f593406dcb0 0x7f59340cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.071+0000 7f5939e00700 1 --2- 172.21.15.3:0/1454524214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59341442c0 0x7f593406d770 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.071+0000 7f5939e00700 1 -- 172.21.15.3:0/1454524214 >> 172.21.15.3:0/1454524214 conn(0x7f59340af620 msgr2=0x7f59340afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.071+0000 7f5939e00700 1 -- 172.21.15.3:0/1454524214 shutdown_connections 2024-09-13T16:52:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.071+0000 7f5939e00700 1 -- 172.21.15.3:0/1454524214 wait complete. 2024-09-13T16:52:57.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 Processor -- start 2024-09-13T16:52:57.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 -- start start 2024-09-13T16:52:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593406dcb0 0x7f5934130180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59340cdd00 0x7f5934140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59341442c0 0x7f59341425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59341306c0 con 0x7f593406dcb0 2024-09-13T16:52:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593413e220 con 0x7f59340cdd00 2024-09-13T16:52:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593413e500 con 0x7f59341442c0 2024-09-13T16:52:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593406dcb0 0x7f5934130180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593406dcb0 0x7f5934130180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39792/0 (socket says 172.21.15.3:39792) 2024-09-13T16:52:57.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 -- 172.21.15.3:0/199998360 learned_addr learned my addr 172.21.15.3:0/199998360 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:57.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5933e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59340cdd00 0x7f5934140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939400700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59341442c0 0x7f59341425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 -- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59341442c0 msgr2=0x7f59341425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59341442c0 0x7f59341425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 -- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59340cdd00 msgr2=0x7f5934140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59340cdd00 0x7f5934140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 -- 172.21.15.3:0/199998360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928009570 con 0x7f593406dcb0 2024-09-13T16:52:57.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5933e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59340cdd00 0x7f5934140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5938a00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593406dcb0 0x7f5934130180 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5928000c00 tx=0x7f5928004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5930c00700 1 -- 172.21.15.3:0/199998360 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928011840 con 0x7f593406dcb0 2024-09-13T16:52:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5930c00700 1 -- 172.21.15.3:0/199998360 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59280119c0 con 0x7f593406dcb0 2024-09-13T16:52:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.075+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593413e7e0 con 0x7f593406dcb0 2024-09-13T16:52:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.079+0000 7f5930c00700 1 -- 172.21.15.3:0/199998360 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928011b70 con 0x7f593406dcb0 2024-09-13T16:52:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.079+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593413eda0 con 0x7f593406dcb0 2024-09-13T16:52:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.079+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593413ef90 con 0x7f593406dcb0 2024-09-13T16:52:57.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.079+0000 7f5930c00700 1 -- 172.21.15.3:0/199998360 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5928011cd0 con 0x7f593406dcb0 2024-09-13T16:52:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.083+0000 7f5930c00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5914066730 0x7f5914068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.083+0000 7f5930c00700 1 -- 172.21.15.3:0/199998360 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5928085530 con 0x7f593406dcb0 2024-09-13T16:52:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.083+0000 7f5933e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5914066730 0x7f5914068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.083+0000 7f5933e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5914066730 0x7f5914068bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f59340b4d10 tx=0x7f5920008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.087+0000 7f5930c00700 1 -- 172.21.15.3:0/199998360 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5928052aa0 con 0x7f593406dcb0 2024-09-13T16:52:57.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.271+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59341446a0 con 0x7f593406dcb0 2024-09-13T16:52:57.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5930c00700 1 -- 172.21.15.3:0/199998360 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5928029020 con 0x7f593406dcb0 2024-09-13T16:52:57.280 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:52:57.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5914066730 msgr2=0x7f5914068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5914066730 0x7f5914068bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f59340b4d10 tx=0x7f5920008040 comp rx=0 tx=0).stop 2024-09-13T16:52:57.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593406dcb0 msgr2=0x7f5934130180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593406dcb0 0x7f5934130180 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5928000c00 tx=0x7f5928004770 comp rx=0 tx=0).stop 2024-09-13T16:52:57.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 shutdown_connections 2024-09-13T16:52:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f5914066730 0x7f5914068bf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59341442c0 0x7f59341425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59340cdd00 0x7f5934140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 --2- 172.21.15.3:0/199998360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593406dcb0 0x7f5934130180 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 >> 172.21.15.3:0/199998360 conn(0x7f59340af620 msgr2=0x7f59340adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.275+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 shutdown_connections 2024-09-13T16:52:57.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.279+0000 7f5939e00700 1 -- 172.21.15.3:0/199998360 wait complete. 2024-09-13T16:52:57.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T16:52:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a86200700 1 Processor -- start 2024-09-13T16:52:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a86200700 1 -- start start 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a86200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7a800cc420 0x7f7a8015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a86200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a800cc800 0x7f7a8006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a86200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8015c160 con 0x7f7a800ce520 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a86200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a80071320 con 0x7f7a800cc420 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a86200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a80071600 con 0x7f7a800cc800 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a7f400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7a800cc420 0x7f7a8015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a7ea00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7a800ce520 0x7f7a80158180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44314/0 (socket says 172.21.15.3:44314) 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a7ea00700 1 -- 172.21.15.3:0/4133496033 learned_addr learned my addr 172.21.15.3:0/4133496033 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a7fe00700 1 --2- 172.21.15.3:0/4133496033 >> v2:172.21.15.157:3300/0 conn(0x7f7a800cc800 0x7f7a8006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a7f400700 1 -- 172.21.15.3:0/4133496033 >> v2:172.21.15.157:3300/0 conn(0x7f7a800cc800 msgr2=0x7f7a8006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a7f400700 1 --2- 172.21.15.3:0/4133496033 >> v2:172.21.15.157:3300/0 conn(0x7f7a800cc800 0x7f7a8006c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.487+0000 7f7a7f400700 1 -- 172.21.15.3:0/4133496033 >> v1:172.21.15.3:6789/0 conn(0x7f7a800ce520 legacy=0x7f7a80158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a7e000700 1 -- 172.21.15.3:0/4133496033 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585899588 0 0) 0x7f7a8015c160 con 0x7f7a800ce520 2024-09-13T16:52:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a7f400700 1 -- 172.21.15.3:0/4133496033 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a800718e0 con 0x7f7a800cc420 2024-09-13T16:52:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a7f400700 1 --2- 172.21.15.3:0/4133496033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc420 0x7f7a8015bc20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7a800cdc40 tx=0x7f7a70009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dbe93317e8d9ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a7e000700 1 -- 172.21.15.3:0/4133496033 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70011830 con 0x7f7a800cc420 2024-09-13T16:52:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a7e000700 1 -- 172.21.15.3:0/4133496033 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a70011990 con 0x7f7a800cc420 2024-09-13T16:52:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a7e000700 1 -- 172.21.15.3:0/4133496033 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70011d40 con 0x7f7a800cc420 2024-09-13T16:52:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a86200700 1 -- 172.21.15.3:0/4133496033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc420 msgr2=0x7f7a8015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a86200700 1 --2- 172.21.15.3:0/4133496033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc420 0x7f7a8015bc20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7a800cdc40 tx=0x7f7a70009b30 comp rx=0 tx=0).stop 2024-09-13T16:52:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a86200700 1 -- 172.21.15.3:0/4133496033 shutdown_connections 2024-09-13T16:52:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a86200700 1 --2- 172.21.15.3:0/4133496033 >> v2:172.21.15.157:3300/0 conn(0x7f7a800cc800 0x7f7a8006c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a86200700 1 --2- 172.21.15.3:0/4133496033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc420 0x7f7a8015bc20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a86200700 1 -- 172.21.15.3:0/4133496033 >> 172.21.15.3:0/4133496033 conn(0x7f7a8006a390 msgr2=0x7f7a8006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:57.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a86200700 1 -- 172.21.15.3:0/4133496033 shutdown_connections 2024-09-13T16:52:57.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.491+0000 7f7a86200700 1 -- 172.21.15.3:0/4133496033 wait complete. 2024-09-13T16:52:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.495+0000 7f7a86200700 1 Processor -- start 2024-09-13T16:52:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.495+0000 7f7a86200700 1 -- start start 2024-09-13T16:52:57.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.495+0000 7f7a86200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a800cc420 0x7f7a8014e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.495+0000 7f7a86200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc800 0x7f7a8014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.495+0000 7f7a86200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a800ce520 0x7f7a800ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.495+0000 7f7a86200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800c8220 con 0x7f7a800cc800 2024-09-13T16:52:57.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.495+0000 7f7a86200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80152600 con 0x7f7a800cc420 2024-09-13T16:52:57.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.495+0000 7f7a86200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801528e0 con 0x7f7a800ce520 2024-09-13T16:52:57.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc800 0x7f7a8014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc800 0x7f7a8014f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39806/0 (socket says 172.21.15.3:39806) 2024-09-13T16:52:57.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 -- 172.21.15.3:0/3687597809 learned_addr learned my addr 172.21.15.3:0/3687597809 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:57.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7f400700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a800cc420 0x7f7a8014e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7fe00700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a800ce520 0x7f7a800ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 -- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a800ce520 msgr2=0x7f7a800ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a800ce520 0x7f7a800ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 -- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a800cc420 msgr2=0x7f7a8014e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a800cc420 0x7f7a8014e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 -- 172.21.15.3:0/3687597809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700094b0 con 0x7f7a800cc800 2024-09-13T16:52:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a7ea00700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc800 0x7f7a8014f360 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00c4e0 tx=0x7f7a6c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a77400700 1 -- 172.21.15.3:0/3687597809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c004ca0 con 0x7f7a800cc800 2024-09-13T16:52:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a77400700 1 -- 172.21.15.3:0/3687597809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c004e20 con 0x7f7a800cc800 2024-09-13T16:52:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a80152bc0 con 0x7f7a800cc800 2024-09-13T16:52:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a77400700 1 -- 172.21.15.3:0/3687597809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c007310 con 0x7f7a800cc800 2024-09-13T16:52:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a80152ea0 con 0x7f7a800cc800 2024-09-13T16:52:57.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.499+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8015bc20 con 0x7f7a800cc800 2024-09-13T16:52:57.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.503+0000 7f7a77400700 1 -- 172.21.15.3:0/3687597809 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a6c007470 con 0x7f7a800cc800 2024-09-13T16:52:57.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.503+0000 7f7a77400700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7a60066730 0x7f7a60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.503+0000 7f7a77400700 1 -- 172.21.15.3:0/3687597809 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7a6c083cf0 con 0x7f7a800cc800 2024-09-13T16:52:57.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.503+0000 7f7a7f400700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7a60066730 0x7f7a60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.507+0000 7f7a7f400700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7a60066730 0x7f7a60068bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7a700090b0 tx=0x7f7a70008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.511+0000 7f7a77400700 1 -- 172.21.15.3:0/3687597809 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a6c051140 con 0x7f7a800cc800 2024-09-13T16:52:57.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.715+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7a8015a160 con 0x7f7a800cc800 2024-09-13T16:52:57.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.715+0000 7f7a77400700 1 -- 172.21.15.3:0/3687597809 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7a6c007720 con 0x7f7a800cc800 2024-09-13T16:52:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:52:57.723 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-13T16:52:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-13T16:52:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-13T16:52:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-13T16:52:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-13T16:52:57.723 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-13T16:52:57.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.719+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7a60066730 msgr2=0x7f7a60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.719+0000 7f7a86200700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7a60066730 0x7f7a60068bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7a700090b0 tx=0x7f7a70008030 comp rx=0 tx=0).stop 2024-09-13T16:52:57.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.719+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc800 msgr2=0x7f7a8014f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.719+0000 7f7a86200700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc800 0x7f7a8014f360 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00c4e0 tx=0x7f7a6c00c7f0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.723+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 shutdown_connections 2024-09-13T16:52:57.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.723+0000 7f7a86200700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7a60066730 0x7f7a60068bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.723+0000 7f7a86200700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a800ce520 0x7f7a800ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.723+0000 7f7a86200700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a800cc800 0x7f7a8014f360 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.723+0000 7f7a86200700 1 --2- 172.21.15.3:0/3687597809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a800cc420 0x7f7a8014e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.723+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 >> 172.21.15.3:0/3687597809 conn(0x7f7a8006a390 msgr2=0x7f7a8015be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:57.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.723+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 shutdown_connections 2024-09-13T16:52:57.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.723+0000 7f7a86200700 1 -- 172.21.15.3:0/3687597809 wait complete. 2024-09-13T16:52:57.750 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T16:52:57.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T16:52:57.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.907+0000 7f9d31400700 1 Processor -- start 2024-09-13T16:52:57.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.907+0000 7f9d31400700 1 -- start start 2024-09-13T16:52:57.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.911+0000 7f9d31400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d2c154ff0 0x7f9d2c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.911+0000 7f9d31400700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0d06d0 con 0x7f9d2c0cd520 2024-09-13T16:52:57.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.911+0000 7f9d31400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c15bc40 con 0x7f9d2c15b770 2024-09-13T16:52:57.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.911+0000 7f9d31400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c15bf20 con 0x7f9d2c154ff0 2024-09-13T16:52:57.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.911+0000 7f9d2be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d2c154ff0 0x7f9d2c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.911+0000 7f9d2be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d2c154ff0 0x7f9d2c15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39814/0 (socket says 172.21.15.3:39814) 2024-09-13T16:52:57.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.911+0000 7f9d2be00700 1 -- 172.21.15.3:0/447806834 learned_addr learned my addr 172.21.15.3:0/447806834 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:57.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d2be00700 1 -- 172.21.15.3:0/447806834 >> v1:172.21.15.157:6789/0 conn(0x7f9d2c15b770 legacy=0x7f9d2c0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:57.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d2be00700 1 -- 172.21.15.3:0/447806834 >> v1:172.21.15.3:6789/0 conn(0x7f9d2c0cd520 legacy=0x7f9d2c15b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d2be00700 1 -- 172.21.15.3:0/447806834 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0b48c0 con 0x7f9d2c154ff0 2024-09-13T16:52:57.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d2be00700 1 --2- 172.21.15.3:0/447806834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c154ff0 0x7f9d2c15c220 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009910 tx=0x7f9d1c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b082ec5a0649b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d2aa00700 1 -- 172.21.15.3:0/447806834 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c004230 con 0x7f9d2c154ff0 2024-09-13T16:52:57.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d2aa00700 1 -- 172.21.15.3:0/447806834 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c004390 con 0x7f9d2c154ff0 2024-09-13T16:52:57.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d2aa00700 1 -- 172.21.15.3:0/447806834 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c009e60 con 0x7f9d2c154ff0 2024-09-13T16:52:57.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d31400700 1 -- 172.21.15.3:0/447806834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c154ff0 msgr2=0x7f9d2c15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d31400700 1 --2- 172.21.15.3:0/447806834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c154ff0 0x7f9d2c15c220 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009910 tx=0x7f9d1c009c20 comp rx=0 tx=0).stop 2024-09-13T16:52:57.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d31400700 1 -- 172.21.15.3:0/447806834 shutdown_connections 2024-09-13T16:52:57.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d31400700 1 --2- 172.21.15.3:0/447806834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c154ff0 0x7f9d2c15c220 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d31400700 1 -- 172.21.15.3:0/447806834 >> 172.21.15.3:0/447806834 conn(0x7f9d2c0b6210 msgr2=0x7f9d2c0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:57.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d31400700 1 -- 172.21.15.3:0/447806834 shutdown_connections 2024-09-13T16:52:57.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.915+0000 7f9d31400700 1 -- 172.21.15.3:0/447806834 wait complete. 2024-09-13T16:52:57.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.919+0000 7f9d31400700 1 Processor -- start 2024-09-13T16:52:57.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.919+0000 7f9d31400700 1 -- start start 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d31400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d2c0cd520 0x7f9d2c14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d31400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d2c154ff0 0x7f9d2c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d31400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c15b770 0x7f9d2c1e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d31400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c1519d0 con 0x7f9d2c15b770 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d31400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c152830 con 0x7f9d2c154ff0 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d31400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c151c20 con 0x7f9d2c0cd520 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c15b770 0x7f9d2c1e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c15b770 0x7f9d2c1e4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39816/0 (socket says 172.21.15.3:39816) 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 -- 172.21.15.3:0/2711925183 learned_addr learned my addr 172.21.15.3:0/2711925183 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d2be00700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d2c0cd520 0x7f9d2c14d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d2b400700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d2c154ff0 0x7f9d2c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 -- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d2c0cd520 msgr2=0x7f9d2c14d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d2c0cd520 0x7f9d2c14d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 -- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d2c154ff0 msgr2=0x7f9d2c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d2c154ff0 0x7f9d2c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 -- 172.21.15.3:0/2711925183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c009570 con 0x7f9d2c15b770 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d30a00700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c15b770 0x7f9d2c1e4490 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b9e0 tx=0x7f9d2400bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d23e00700 1 -- 172.21.15.3:0/2711925183 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24003c20 con 0x7f9d2c15b770 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d23e00700 1 -- 172.21.15.3:0/2711925183 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24003dc0 con 0x7f9d2c15b770 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d23e00700 1 -- 172.21.15.3:0/2711925183 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400d920 con 0x7f9d2c15b770 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c151f00 con 0x7f9d2c15b770 2024-09-13T16:52:57.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.923+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c1524c0 con 0x7f9d2c15b770 2024-09-13T16:52:57.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.927+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c1529a0 con 0x7f9d2c15b770 2024-09-13T16:52:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.927+0000 7f9d23e00700 1 -- 172.21.15.3:0/2711925183 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d2400da80 con 0x7f9d2c15b770 2024-09-13T16:52:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.927+0000 7f9d23e00700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9d0c066760 0x7f9d0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.931+0000 7f9d23e00700 1 -- 172.21.15.3:0/2711925183 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9d24011070 con 0x7f9d2c15b770 2024-09-13T16:52:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.931+0000 7f9d2be00700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9d0c066760 0x7f9d0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.931+0000 7f9d2be00700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9d0c066760 0x7f9d0c068c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009910 tx=0x7f9d1c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:57.935+0000 7f9d23e00700 1 -- 172.21.15.3:0/2711925183 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9d24051330 con 0x7f9d2c15b770 2024-09-13T16:52:58.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.115+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9d2c15c880 con 0x7f9d2c15b770 2024-09-13T16:52:58.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.587+0000 7f9d23e00700 1 -- 172.21.15.3:0/2711925183 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9d24054950 con 0x7f9d2c15b770 2024-09-13T16:52:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9d0c066760 msgr2=0x7f9d0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9d0c066760 0x7f9d0c068c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009910 tx=0x7f9d1c008030 comp rx=0 tx=0).stop 2024-09-13T16:52:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c15b770 msgr2=0x7f9d2c1e4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c15b770 0x7f9d2c1e4490 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b9e0 tx=0x7f9d2400bcf0 comp rx=0 tx=0).stop 2024-09-13T16:52:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 shutdown_connections 2024-09-13T16:52:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9d0c066760 0x7f9d0c068c20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c15b770 0x7f9d2c1e4490 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d2c154ff0 0x7f9d2c152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 --2- 172.21.15.3:0/2711925183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d2c0cd520 0x7f9d2c14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 >> 172.21.15.3:0/2711925183 conn(0x7f9d2c0b6210 msgr2=0x7f9d2c15ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 shutdown_connections 2024-09-13T16:52:58.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.599+0000 7f9d31400700 1 -- 172.21.15.3:0/2711925183 wait complete. 2024-09-13T16:52:58.608 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-13T16:52:58.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T16:52:58.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:58.809 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.811+0000 7fbde2c00700 1 Processor -- start 2024-09-13T16:52:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.811+0000 7fbde2c00700 1 -- start start 2024-09-13T16:52:58.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbde2c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbddc0cdc30 0x7fbddc15b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:58.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbde2c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc0b41e0 con 0x7fbddc1442c0 2024-09-13T16:52:58.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbde2c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc06a390 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbde2c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc06b1f0 con 0x7fbddc0cdc30 2024-09-13T16:52:58.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbde1800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbddc1442c0 0x7fbddc06d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44322/0 (socket says 172.21.15.3:44322) 2024-09-13T16:52:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbde1800700 1 -- 172.21.15.3:0/987122231 learned_addr learned my addr 172.21.15.3:0/987122231 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbde2200700 1 --2- 172.21.15.3:0/987122231 >> v2:172.21.15.157:3300/0 conn(0x7fbddc0cdc30 0x7fbddc15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:58.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbddbe00700 1 -- 172.21.15.3:0/987122231 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111565671 0 0) 0x7fbddc0b41e0 con 0x7fbddc1442c0 2024-09-13T16:52:58.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbddbe00700 1 -- 172.21.15.3:0/987122231 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc40035d0 con 0x7fbddc1442c0 2024-09-13T16:52:58.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.815+0000 7fbddbe00700 1 -- 172.21.15.3:0/987122231 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077315786 0 0) 0x7fbddc06a390 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbddbe00700 1 -- 172.21.15.3:0/987122231 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbddc0b41e0 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2200700 1 -- 172.21.15.3:0/987122231 >> v1:172.21.15.157:6789/0 conn(0x7fbddc0cd7c0 legacy=0x7fbddc15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:58.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2200700 1 -- 172.21.15.3:0/987122231 >> v1:172.21.15.3:6789/0 conn(0x7fbddc1442c0 legacy=0x7fbddc06d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:58.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2200700 1 -- 172.21.15.3:0/987122231 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc06a5e0 con 0x7fbddc0cdc30 2024-09-13T16:52:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2200700 1 --2- 172.21.15.3:0/987122231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc0cdc30 0x7fbddc15b820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009d90 tx=0x7fbdd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=300da5a05fb3b67c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbddbe00700 1 -- 172.21.15.3:0/987122231 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400e070 con 0x7fbddc0cdc30 2024-09-13T16:52:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbddbe00700 1 -- 172.21.15.3:0/987122231 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd400bb20 con 0x7fbddc0cdc30 2024-09-13T16:52:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbddbe00700 1 -- 172.21.15.3:0/987122231 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4012430 con 0x7fbddc0cdc30 2024-09-13T16:52:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2c00700 1 -- 172.21.15.3:0/987122231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc0cdc30 msgr2=0x7fbddc15b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2c00700 1 --2- 172.21.15.3:0/987122231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc0cdc30 0x7fbddc15b820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009d90 tx=0x7fbdd400b330 comp rx=0 tx=0).stop 2024-09-13T16:52:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2c00700 1 -- 172.21.15.3:0/987122231 shutdown_connections 2024-09-13T16:52:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2c00700 1 --2- 172.21.15.3:0/987122231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc0cdc30 0x7fbddc15b820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2c00700 1 -- 172.21.15.3:0/987122231 >> 172.21.15.3:0/987122231 conn(0x7fbddc0af620 msgr2=0x7fbddc0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2c00700 1 -- 172.21.15.3:0/987122231 shutdown_connections 2024-09-13T16:52:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.819+0000 7fbde2c00700 1 -- 172.21.15.3:0/987122231 wait complete. 2024-09-13T16:52:58.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.823+0000 7fbde2c00700 1 Processor -- start 2024-09-13T16:52:58.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.823+0000 7fbde2c00700 1 -- start start 2024-09-13T16:52:58.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cd7c0 0x7fbddc142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:58.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbddc0cdc30 0x7fbddc13b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:58.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc1442c0 0x7fbddc1f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:58.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc13dc80 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc13df60 con 0x7fbddc1442c0 2024-09-13T16:52:58.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc13e240 con 0x7fbddc0cdc30 2024-09-13T16:52:58.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cd7c0 0x7fbddc142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cd7c0 0x7fbddc142b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39826/0 (socket says 172.21.15.3:39826) 2024-09-13T16:52:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 -- 172.21.15.3:0/3961209243 learned_addr learned my addr 172.21.15.3:0/3961209243 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde0e00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbddc0cdc30 0x7fbddc13b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 -- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbddc0cdc30 msgr2=0x7fbddc13b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbddc0cdc30 0x7fbddc13b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 -- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc1442c0 msgr2=0x7fbddc1f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2200700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc1442c0 0x7fbddc1f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc1442c0 0x7fbddc1f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 -- 172.21.15.3:0/3961209243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4009a70 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2200700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc1442c0 0x7fbddc1f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde1800700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cd7c0 0x7fbddc142b80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000c580 tx=0x7fbdd000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbdd9600700 1 -- 172.21.15.3:0/3961209243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0005d90 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc13e520 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc13bed0 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbdd9600700 1 -- 172.21.15.3:0/3961209243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd00070d0 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.827+0000 7fbdd9600700 1 -- 172.21.15.3:0/3961209243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0007440 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.831+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc13ea40 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.831+0000 7fbdd9600700 1 -- 172.21.15.3:0/3961209243 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbdd00075a0 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.835+0000 7fbdd9600700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbdbc066730 0x7fbdbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:58.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.835+0000 7fbdd9600700 1 -- 172.21.15.3:0/3961209243 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fbdd0083b60 con 0x7fbddc0cd7c0 2024-09-13T16:52:58.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.835+0000 7fbde0e00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbdbc066730 0x7fbdbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:58.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.835+0000 7fbde0e00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbdbc066730 0x7fbdbc068bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8007fd0 tx=0x7fbdc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:58.839+0000 7fbdd9600700 1 -- 172.21.15.3:0/3961209243 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbdd0051090 con 0x7fbddc0cd7c0 2024-09-13T16:52:59.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.023+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbddc1446a0 con 0x7fbddc0cd7c0 2024-09-13T16:52:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.607+0000 7fbdd9600700 1 -- 172.21.15.3:0/3961209243 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbdd00170d0 con 0x7fbddc0cd7c0 2024-09-13T16:52:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.619+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbdbc066730 msgr2=0x7fbdbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.619+0000 7fbde2c00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbdbc066730 0x7fbdbc068bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8007fd0 tx=0x7fbdc8009040 comp rx=0 tx=0).stop 2024-09-13T16:52:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.619+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cd7c0 msgr2=0x7fbddc142b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.619+0000 7fbde2c00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cd7c0 0x7fbddc142b80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000c580 tx=0x7fbdd000c940 comp rx=0 tx=0).stop 2024-09-13T16:52:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.623+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 shutdown_connections 2024-09-13T16:52:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.623+0000 7fbde2c00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbdbc066730 0x7fbdbc068bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.623+0000 7fbde2c00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbddc1442c0 0x7fbddc1f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.623+0000 7fbde2c00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbddc0cdc30 0x7fbddc13b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.623+0000 7fbde2c00700 1 --2- 172.21.15.3:0/3961209243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cd7c0 0x7fbddc142b80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.623+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 >> 172.21.15.3:0/3961209243 conn(0x7fbddc0af620 msgr2=0x7fbddc0b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.627+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 shutdown_connections 2024-09-13T16:52:59.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.627+0000 7fbde2c00700 1 -- 172.21.15.3:0/3961209243 wait complete. 2024-09-13T16:52:59.636 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-13T16:52:59.660 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T16:52:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.851+0000 7fcdee800700 1 Processor -- start 2024-09-13T16:52:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.851+0000 7fcdee800700 1 -- start start 2024-09-13T16:52:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.851+0000 7fcdee800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcdee800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcde806dac0 0x7fcde80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcdee800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80b1f30 con 0x7fcde80b3be0 2024-09-13T16:52:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcdee800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80aca50 con 0x7fcde806dac0 2024-09-13T16:52:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcdee800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80723e0 con 0x7fcde80b3800 2024-09-13T16:52:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde7e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcde806dac0 0x7fcde80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde6a00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcde80b3be0 0x7fcde806d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:50850/0 (socket says 172.21.15.3:50850) 2024-09-13T16:52:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde6a00700 1 -- 172.21.15.3:0/874043210 learned_addr learned my addr 172.21.15.3:0/874043210 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:52:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde7400700 1 --2- 172.21.15.3:0/874043210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde6000700 1 -- 172.21.15.3:0/874043210 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007630547 0 0) 0x7fcde80b1f30 con 0x7fcde80b3be0 2024-09-13T16:52:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde6000700 1 -- 172.21.15.3:0/874043210 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdcc0035d0 con 0x7fcde80b3be0 2024-09-13T16:52:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde7400700 1 -- 172.21.15.3:0/874043210 >> v2:172.21.15.157:3300/0 conn(0x7fcde806dac0 msgr2=0x7fcde80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde7400700 1 --2- 172.21.15.3:0/874043210 >> v2:172.21.15.157:3300/0 conn(0x7fcde806dac0 0x7fcde80714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde7400700 1 -- 172.21.15.3:0/874043210 >> v1:172.21.15.157:6789/0 conn(0x7fcde80b3be0 legacy=0x7fcde806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde7400700 1 -- 172.21.15.3:0/874043210 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde8071b40 con 0x7fcde80b3800 2024-09-13T16:52:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde7400700 1 --2- 172.21.15.3:0/874043210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8070f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009d90 tx=0x7fcdd800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f257f5a510243b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde6000700 1 -- 172.21.15.3:0/874043210 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800e070 con 0x7fcde80b3800 2024-09-13T16:52:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde6000700 1 -- 172.21.15.3:0/874043210 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd800bb20 con 0x7fcde80b3800 2024-09-13T16:52:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.855+0000 7fcde6000700 1 -- 172.21.15.3:0/874043210 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8012430 con 0x7fcde80b3800 2024-09-13T16:52:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.859+0000 7fcdee800700 1 -- 172.21.15.3:0/874043210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 msgr2=0x7fcde8070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.859+0000 7fcdee800700 1 --2- 172.21.15.3:0/874043210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8070f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009d90 tx=0x7fcdd800b330 comp rx=0 tx=0).stop 2024-09-13T16:52:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.859+0000 7fcdee800700 1 -- 172.21.15.3:0/874043210 shutdown_connections 2024-09-13T16:52:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.859+0000 7fcdee800700 1 --2- 172.21.15.3:0/874043210 >> v2:172.21.15.157:3300/0 conn(0x7fcde806dac0 0x7fcde80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.859+0000 7fcdee800700 1 --2- 172.21.15.3:0/874043210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8070f80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.859+0000 7fcdee800700 1 -- 172.21.15.3:0/874043210 >> 172.21.15.3:0/874043210 conn(0x7fcde806b280 msgr2=0x7fcde806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.859+0000 7fcdee800700 1 -- 172.21.15.3:0/874043210 shutdown_connections 2024-09-13T16:52:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.859+0000 7fcdee800700 1 -- 172.21.15.3:0/874043210 wait complete. 2024-09-13T16:52:59.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcdee800700 1 Processor -- start 2024-09-13T16:52:59.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcdee800700 1 -- start start 2024-09-13T16:52:59.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcdee800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde806dac0 0x7fcde80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcdee800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcdee800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde80b3be0 0x7fcde80cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcdee800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde81486c0 con 0x7fcde806dac0 2024-09-13T16:52:59.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcdee800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80cbdc0 con 0x7fcde80b3be0 2024-09-13T16:52:59.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcdee800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80cc0a0 con 0x7fcde80b3800 2024-09-13T16:52:59.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcde7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde806dac0 0x7fcde80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcde7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde806dac0 0x7fcde80cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39840/0 (socket says 172.21.15.3:39840) 2024-09-13T16:52:59.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.863+0000 7fcde7400700 1 -- 172.21.15.3:0/2239593750 learned_addr learned my addr 172.21.15.3:0/2239593750 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:52:59.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcde7e00700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde80b3be0 0x7fcde80cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcde6a00700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcde7400700 1 -- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 msgr2=0x7fcde8148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcde7400700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcde7400700 1 -- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde80b3be0 msgr2=0x7fcde80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcde7400700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde80b3be0 0x7fcde80cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcde7400700 1 -- 172.21.15.3:0/2239593750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8009a70 con 0x7fcde806dac0 2024-09-13T16:52:59.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcde7400700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde806dac0 0x7fcde80cce60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8014040 tx=0x7fcdd8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcddf400700 1 -- 172.21.15.3:0/2239593750 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8012430 con 0x7fcde806dac0 2024-09-13T16:52:59.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcddf400700 1 -- 172.21.15.3:0/2239593750 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8020070 con 0x7fcde806dac0 2024-09-13T16:52:59.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde80cc380 con 0x7fcde806dac0 2024-09-13T16:52:59.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcddf400700 1 -- 172.21.15.3:0/2239593750 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8004280 con 0x7fcde806dac0 2024-09-13T16:52:59.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.867+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde80cc940 con 0x7fcde806dac0 2024-09-13T16:52:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.871+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde806bee0 con 0x7fcde806dac0 2024-09-13T16:52:59.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.871+0000 7fcddf400700 1 -- 172.21.15.3:0/2239593750 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcdd80043e0 con 0x7fcde806dac0 2024-09-13T16:52:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.875+0000 7fcddf400700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcdc8066730 0x7fcdc8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.875+0000 7fcddf400700 1 -- 172.21.15.3:0/2239593750 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fcdd8022070 con 0x7fcde806dac0 2024-09-13T16:52:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.875+0000 7fcde6a00700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcdc8066730 0x7fcdc8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.875+0000 7fcde6a00700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcdc8066730 0x7fcdc8068bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcde80b0770 tx=0x7fcdd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:52:59.883+0000 7fcddf400700 1 -- 172.21.15.3:0/2239593750 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcdd8053410 con 0x7fcde806dac0 2024-09-13T16:53:00.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.107+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcde806d3b0 con 0x7fcde806dac0 2024-09-13T16:53:00.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.123+0000 7fcddf400700 1 -- 172.21.15.3:0/2239593750 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v26) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcdd8056a30 con 0x7fcde806dac0 2024-09-13T16:53:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcdc8066730 msgr2=0x7fcdc8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcdc8066730 0x7fcdc8068bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcde80b0770 tx=0x7fcdd4009040 comp rx=0 tx=0).stop 2024-09-13T16:53:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde806dac0 msgr2=0x7fcde80cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde806dac0 0x7fcde80cce60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8014040 tx=0x7fcdd8004af0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 shutdown_connections 2024-09-13T16:53:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcdc8066730 0x7fcdc8068bf0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcde80b0770 tx=0x7fcdd4009040 comp rx=0 tx=0).stop 2024-09-13T16:53:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde80b3be0 0x7fcde80cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80b3800 0x7fcde8148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 --2- 172.21.15.3:0/2239593750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde806dac0 0x7fcde80cce60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.135+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 >> 172.21.15.3:0/2239593750 conn(0x7fcde806b280 msgr2=0x7fcde80af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:00.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.143+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 shutdown_connections 2024-09-13T16:53:00.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.143+0000 7fcdee800700 1 -- 172.21.15.3:0/2239593750 wait complete. 2024-09-13T16:53:00.155 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 7 and data pool 8 2024-09-13T16:53:00.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:53:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.347+0000 7f75e1200700 1 Processor -- start 2024-09-13T16:53:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.347+0000 7f75e1200700 1 -- start start 2024-09-13T16:53:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.347+0000 7f75e1200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06a390 0x7f75dc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.347+0000 7f75e1200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75dc15e720 0x7f75dc15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.347+0000 7f75e1200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc06e360 con 0x7f75dc15cd40 2024-09-13T16:53:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.347+0000 7f75e1200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc15e130 con 0x7f75dc15e720 2024-09-13T16:53:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.347+0000 7f75e1200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc06ad10 con 0x7f75dc06a390 2024-09-13T16:53:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75dbe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75dc15e720 0x7f75dc15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75dbe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75dc15e720 0x7f75dc15ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39852/0 (socket says 172.21.15.3:39852) 2024-09-13T16:53:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75dbe00700 1 -- 172.21.15.3:0/3054884130 learned_addr learned my addr 172.21.15.3:0/3054884130 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75d2a00700 1 --2- 172.21.15.3:0/3054884130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06a390 0x7f75dc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75dbe00700 1 -- 172.21.15.3:0/3054884130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06a390 msgr2=0x7f75dc06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:00.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75dbe00700 1 --2- 172.21.15.3:0/3054884130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06a390 0x7f75dc06a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75dbe00700 1 -- 172.21.15.3:0/3054884130 >> v1:172.21.15.157:6789/0 conn(0x7f75dc15cd40 legacy=0x7f75dc1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75dbe00700 1 -- 172.21.15.3:0/3054884130 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc06d4b0 con 0x7f75dc15e720 2024-09-13T16:53:00.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75daa00700 1 -- 172.21.15.3:0/3054884130 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734820089 0 0) 0x7f75dc06e360 con 0x7f75dc15cd40 2024-09-13T16:53:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75dbe00700 1 --2- 172.21.15.3:0/3054884130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc15e720 0x7f75dc15ebd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009e50 tx=0x7f75d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c72883bb4d328b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75daa00700 1 -- 172.21.15.3:0/3054884130 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400e070 con 0x7f75dc15e720 2024-09-13T16:53:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75daa00700 1 -- 172.21.15.3:0/3054884130 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d400b9c0 con 0x7f75dc15e720 2024-09-13T16:53:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.351+0000 7f75daa00700 1 -- 172.21.15.3:0/3054884130 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400bd70 con 0x7f75dc15e720 2024-09-13T16:53:00.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 -- 172.21.15.3:0/3054884130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc15e720 msgr2=0x7f75dc15ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 --2- 172.21.15.3:0/3054884130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc15e720 0x7f75dc15ebd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009e50 tx=0x7f75d400b330 comp rx=0 tx=0).stop 2024-09-13T16:53:00.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 -- 172.21.15.3:0/3054884130 shutdown_connections 2024-09-13T16:53:00.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 --2- 172.21.15.3:0/3054884130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc15e720 0x7f75dc15ebd0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 --2- 172.21.15.3:0/3054884130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06a390 0x7f75dc06a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 -- 172.21.15.3:0/3054884130 >> 172.21.15.3:0/3054884130 conn(0x7f75dc071d80 msgr2=0x7f75dc06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:00.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 -- 172.21.15.3:0/3054884130 shutdown_connections 2024-09-13T16:53:00.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 -- 172.21.15.3:0/3054884130 wait complete. 2024-09-13T16:53:00.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 Processor -- start 2024-09-13T16:53:00.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.355+0000 7f75e1200700 1 -- start start 2024-09-13T16:53:00.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75e1200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc06a390 0x7f75dc1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75e1200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc15cd40 0x7f75dc0cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75e1200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc15e720 0x7f75dc155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75e1200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0ceca0 con 0x7f75dc06a390 2024-09-13T16:53:00.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75e1200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0ce090 con 0x7f75dc15cd40 2024-09-13T16:53:00.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75e1200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0ce300 con 0x7f75dc15e720 2024-09-13T16:53:00.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc06a390 0x7f75dc1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc06a390 0x7f75dc1518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39854/0 (socket says 172.21.15.3:39854) 2024-09-13T16:53:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 -- 172.21.15.3:0/108078149 learned_addr learned my addr 172.21.15.3:0/108078149 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75dbe00700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc15e720 0x7f75dc155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75d2a00700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc15cd40 0x7f75dc0cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 -- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc15e720 msgr2=0x7f75dc155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc15e720 0x7f75dc155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 -- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc15cd40 msgr2=0x7f75dc0cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc15cd40 0x7f75dc0cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 -- 172.21.15.3:0/108078149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4009b30 con 0x7f75dc06a390 2024-09-13T16:53:00.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75db400700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc06a390 0x7f75dc1518d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c520 tx=0x7f75cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:00.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75d3e00700 1 -- 172.21.15.3:0/108078149 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0054e0 con 0x7f75dc06a390 2024-09-13T16:53:00.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75d3e00700 1 -- 172.21.15.3:0/108078149 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc0036a0 con 0x7f75dc06a390 2024-09-13T16:53:00.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75d3e00700 1 -- 172.21.15.3:0/108078149 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc003a50 con 0x7f75dc06a390 2024-09-13T16:53:00.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc0ce5e0 con 0x7f75dc06a390 2024-09-13T16:53:00.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.359+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc0cf600 con 0x7f75dc06a390 2024-09-13T16:53:00.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.363+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc06f160 con 0x7f75dc06a390 2024-09-13T16:53:00.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.363+0000 7f75d3e00700 1 -- 172.21.15.3:0/108078149 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75cc003bb0 con 0x7f75dc06a390 2024-09-13T16:53:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.367+0000 7f75d3e00700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f75b8066730 0x7f75b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.367+0000 7f75d3e00700 1 -- 172.21.15.3:0/108078149 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f75cc083650 con 0x7f75dc06a390 2024-09-13T16:53:00.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.367+0000 7f75d2a00700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f75b8066730 0x7f75b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.367+0000 7f75d2a00700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f75b8066730 0x7f75b8068bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0adf40 tx=0x7f75c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:00.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.371+0000 7f75d3e00700 1 -- 172.21.15.3:0/108078149 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f75cc050870 con 0x7f75dc06a390 2024-09-13T16:53:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.571+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75dc06ed80 con 0x7f75dc06a390 2024-09-13T16:53:00.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.575+0000 7f75d3e00700 1 -- 172.21.15.3:0/108078149 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16739 (secure 0 0 0) 0x7f75cc003e60 con 0x7f75dc06a390 2024-09-13T16:53:00.581 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:00.582 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":57,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:53:00.113006+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:58.182202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:59.098263+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:53:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f75b8066730 msgr2=0x7f75b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f75b8066730 0x7f75b8068bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0adf40 tx=0x7f75c4008040 comp rx=0 tx=0).stop 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc06a390 msgr2=0x7f75dc1518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc06a390 0x7f75dc1518d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c520 tx=0x7f75cc00c830 comp rx=0 tx=0).stop 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 shutdown_connections 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f75b8066730 0x7f75b8068bf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc15e720 0x7f75dc155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc15cd40 0x7f75dc0cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 --2- 172.21.15.3:0/108078149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc06a390 0x7f75dc1518d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 >> 172.21.15.3:0/108078149 conn(0x7f75dc071d80 msgr2=0x7f75dc15eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 shutdown_connections 2024-09-13T16:53:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.579+0000 7f75e1200700 1 -- 172.21.15.3:0/108078149 wait complete. 2024-09-13T16:53:00.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:53:00.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.775+0000 7f71ae400700 1 Processor -- start 2024-09-13T16:53:00.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71ae400700 1 -- start start 2024-09-13T16:53:00.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71ae400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 0x7f71a815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71ae400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71a80ce520 0x7f71a8158180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71ae400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71a80cc800 0x7f71a81586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71ae400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a8158c00 con 0x7f71a80cc800 2024-09-13T16:53:00.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71ae400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a815c160 con 0x7f71a80ce520 2024-09-13T16:53:00.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71ae400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8071320 con 0x7f71a80cc420 2024-09-13T16:53:00.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71a7e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71a80cc800 0x7f71a81586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71a7e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71a80cc800 0x7f71a81586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39858/0 (socket says 172.21.15.3:39858) 2024-09-13T16:53:00.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71a7e00700 1 -- 172.21.15.3:0/2183275402 learned_addr learned my addr 172.21.15.3:0/2183275402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:00.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71a6a00700 1 --2- 172.21.15.3:0/2183275402 >> v2:172.21.15.157:3300/0 conn(0x7f71a80ce520 0x7f71a8158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71a7400700 1 --2- 172.21.15.3:0/2183275402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 0x7f71a815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.779+0000 7f71a7e00700 1 -- 172.21.15.3:0/2183275402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 msgr2=0x7f71a815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:00.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a7e00700 1 --2- 172.21.15.3:0/2183275402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 0x7f71a815bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a7e00700 1 -- 172.21.15.3:0/2183275402 >> v2:172.21.15.157:3300/0 conn(0x7f71a80ce520 msgr2=0x7f71a8158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:00.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a7e00700 1 --2- 172.21.15.3:0/2183275402 >> v2:172.21.15.157:3300/0 conn(0x7f71a80ce520 0x7f71a8158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a7e00700 1 -- 172.21.15.3:0/2183275402 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a8071600 con 0x7f71a80cc800 2024-09-13T16:53:00.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a6a00700 1 --2- 172.21.15.3:0/2183275402 >> v2:172.21.15.157:3300/0 conn(0x7f71a80ce520 0x7f71a8158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:00.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a7e00700 1 --2- 172.21.15.3:0/2183275402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81586c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f71a806e090 tx=0x7f71a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30de584d6775e454 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:00.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a6000700 1 -- 172.21.15.3:0/2183275402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000e070 con 0x7f71a80cc800 2024-09-13T16:53:00.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a6000700 1 -- 172.21.15.3:0/2183275402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a000b9c0 con 0x7f71a80cc800 2024-09-13T16:53:00.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71a6000700 1 -- 172.21.15.3:0/2183275402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000bd70 con 0x7f71a80cc800 2024-09-13T16:53:00.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 -- 172.21.15.3:0/2183275402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 msgr2=0x7f71a81586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 --2- 172.21.15.3:0/2183275402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81586c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f71a806e090 tx=0x7f71a000b330 comp rx=0 tx=0).stop 2024-09-13T16:53:00.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 -- 172.21.15.3:0/2183275402 shutdown_connections 2024-09-13T16:53:00.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 --2- 172.21.15.3:0/2183275402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81586c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 --2- 172.21.15.3:0/2183275402 >> v2:172.21.15.157:3300/0 conn(0x7f71a80ce520 0x7f71a8158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 --2- 172.21.15.3:0/2183275402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 0x7f71a815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 -- 172.21.15.3:0/2183275402 >> 172.21.15.3:0/2183275402 conn(0x7f71a806a390 msgr2=0x7f71a806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 -- 172.21.15.3:0/2183275402 shutdown_connections 2024-09-13T16:53:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.783+0000 7f71ae400700 1 -- 172.21.15.3:0/2183275402 wait complete. 2024-09-13T16:53:00.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71ae400700 1 Processor -- start 2024-09-13T16:53:00.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71ae400700 1 -- start start 2024-09-13T16:53:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71ae400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 0x7f71a80c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71ae400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81f0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71ae400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71a80ce520 0x7f71a81f0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71ae400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c4530 con 0x7f71a80cc800 2024-09-13T16:53:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71ae400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c4810 con 0x7f71a80ce520 2024-09-13T16:53:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71ae400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c4af0 con 0x7f71a80cc420 2024-09-13T16:53:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.787+0000 7f71a6a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a7400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 0x7f71a80c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a6a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81f0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39860/0 (socket says 172.21.15.3:39860) 2024-09-13T16:53:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a6a00700 1 -- 172.21.15.3:0/503844466 learned_addr learned my addr 172.21.15.3:0/503844466 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a7e00700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71a80ce520 0x7f71a81f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a6a00700 1 -- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 msgr2=0x7f71a80c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a6a00700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 0x7f71a80c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a6a00700 1 -- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71a80ce520 msgr2=0x7f71a81f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a6a00700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71a80ce520 0x7f71a81f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a6a00700 1 -- 172.21.15.3:0/503844466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0009b30 con 0x7f71a80cc800 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71a6a00700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81f0250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f719400e810 tx=0x7f719400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f719f400700 1 -- 172.21.15.3:0/503844466 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194004230 con 0x7f71a80cc800 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f719f400700 1 -- 172.21.15.3:0/503844466 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7194004390 con 0x7f71a80cc800 2024-09-13T16:53:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f719f400700 1 -- 172.21.15.3:0/503844466 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940036c0 con 0x7f71a80cc800 2024-09-13T16:53:00.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a80c4dd0 con 0x7f71a80cc800 2024-09-13T16:53:00.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.791+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a80c4220 con 0x7f71a80cc800 2024-09-13T16:53:00.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.795+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a8151ed0 con 0x7f71a80cc800 2024-09-13T16:53:00.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.795+0000 7f719f400700 1 -- 172.21.15.3:0/503844466 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7194003840 con 0x7f71a80cc800 2024-09-13T16:53:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.799+0000 7f719f400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7188066730 0x7f7188068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:00.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.799+0000 7f719f400700 1 -- 172.21.15.3:0/503844466 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7194083ba0 con 0x7f71a80cc800 2024-09-13T16:53:00.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.799+0000 7f71a7400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7188066730 0x7f7188068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:00.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.799+0000 7f71a7400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7188066730 0x7f7188068bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7198010e90 tx=0x7f7198012040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:00.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.803+0000 7f719f400700 1 -- 172.21.15.3:0/503844466 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7194050dc0 con 0x7f71a80cc800 2024-09-13T16:53:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:00.999+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71a815bc00 con 0x7f71a80cc800 2024-09-13T16:53:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.003+0000 7f719f400700 1 -- 172.21.15.3:0/503844466 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16739 (secure 0 0 0) 0x7f7194020090 con 0x7f71a80cc800 2024-09-13T16:53:01.009 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:01.010 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":57,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:53:00.113006+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:58.182202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:59.098263+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:53:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7188066730 msgr2=0x7f7188068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7188066730 0x7f7188068bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7198010e90 tx=0x7f7198012040 comp rx=0 tx=0).stop 2024-09-13T16:53:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 msgr2=0x7f71a81f0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81f0250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f719400e810 tx=0x7f719400eb20 comp rx=0 tx=0).stop 2024-09-13T16:53:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 shutdown_connections 2024-09-13T16:53:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7188066730 0x7f7188068bf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71a80ce520 0x7f71a81f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80cc800 0x7f71a81f0250 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 --2- 172.21.15.3:0/503844466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71a80cc420 0x7f71a80c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 >> 172.21.15.3:0/503844466 conn(0x7f71a806a390 msgr2=0x7f71a8157300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 shutdown_connections 2024-09-13T16:53:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.007+0000 7f71ae400700 1 -- 172.21.15.3:0/503844466 wait complete. 2024-09-13T16:53:01.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T16:53:01.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.215+0000 7f8826600700 1 Processor -- start 2024-09-13T16:53:01.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.215+0000 7f8826600700 1 -- start start 2024-09-13T16:53:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88200ccbe0 0x7f88200ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88200cc530 0x7f88200cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88200cea90 con 0x7f88200b3e20 2024-09-13T16:53:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88200b3a00 con 0x7f88200ccbe0 2024-09-13T16:53:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88200b3410 con 0x7f88200cc530 2024-09-13T16:53:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881f400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88200ccbe0 0x7f88200ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881fe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f88200b3e20 0x7f882015adf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44338/0 (socket says 172.21.15.3:44338) 2024-09-13T16:53:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881fe00700 1 -- 172.21.15.3:0/4291927016 learned_addr learned my addr 172.21.15.3:0/4291927016 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881ea00700 1 --2- 172.21.15.3:0/4291927016 >> v2:172.21.15.157:3300/0 conn(0x7f88200cc530 0x7f88200cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881f400700 1 -- 172.21.15.3:0/4291927016 >> v2:172.21.15.157:3300/0 conn(0x7f88200cc530 msgr2=0x7f88200cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881f400700 1 --2- 172.21.15.3:0/4291927016 >> v2:172.21.15.157:3300/0 conn(0x7f88200cc530 0x7f88200cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881f400700 1 -- 172.21.15.3:0/4291927016 >> v1:172.21.15.3:6789/0 conn(0x7f88200b3e20 legacy=0x7f882015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881f400700 1 -- 172.21.15.3:0/4291927016 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88200cd500 con 0x7f88200ccbe0 2024-09-13T16:53:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881f400700 1 --2- 172.21.15.3:0/4291927016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88200ccfc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f88200720f0 tx=0x7f8810009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc839f2258839da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881e000700 1 -- 172.21.15.3:0/4291927016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810011830 con 0x7f88200ccbe0 2024-09-13T16:53:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881e000700 1 -- 172.21.15.3:0/4291927016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8810011990 con 0x7f88200ccbe0 2024-09-13T16:53:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f881e000700 1 -- 172.21.15.3:0/4291927016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810011d40 con 0x7f88200ccbe0 2024-09-13T16:53:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 -- 172.21.15.3:0/4291927016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 msgr2=0x7f88200ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 --2- 172.21.15.3:0/4291927016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88200ccfc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f88200720f0 tx=0x7f8810009b30 comp rx=0 tx=0).stop 2024-09-13T16:53:01.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 -- 172.21.15.3:0/4291927016 shutdown_connections 2024-09-13T16:53:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 --2- 172.21.15.3:0/4291927016 >> v2:172.21.15.157:3300/0 conn(0x7f88200cc530 0x7f88200cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 --2- 172.21.15.3:0/4291927016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88200ccfc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.219+0000 7f8826600700 1 -- 172.21.15.3:0/4291927016 >> 172.21.15.3:0/4291927016 conn(0x7f8820071680 msgr2=0x7f882006a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.223+0000 7f8826600700 1 -- 172.21.15.3:0/4291927016 shutdown_connections 2024-09-13T16:53:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.223+0000 7f8826600700 1 -- 172.21.15.3:0/4291927016 wait complete. 2024-09-13T16:53:01.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.223+0000 7f8826600700 1 Processor -- start 2024-09-13T16:53:01.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f8826600700 1 -- start start 2024-09-13T16:53:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f8826600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200b3e20 0x7f88201e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f8826600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88200cc530 0x7f8820147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f8826600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88201e4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f8826600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88201480d0 con 0x7f88200ccbe0 2024-09-13T16:53:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f8826600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882006caa0 con 0x7f88200cc530 2024-09-13T16:53:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f8826600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882006cd80 con 0x7f88200b3e20 2024-09-13T16:53:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88201e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88201e4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39886/0 (socket says 172.21.15.3:39886) 2024-09-13T16:53:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 -- 172.21.15.3:0/1444316704 learned_addr learned my addr 172.21.15.3:0/1444316704 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881f400700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200b3e20 0x7f88201e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881ea00700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88200cc530 0x7f8820147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 -- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200b3e20 msgr2=0x7f88201e25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200b3e20 0x7f88201e25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 -- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88200cc530 msgr2=0x7f8820147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88200cc530 0x7f8820147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 -- 172.21.15.3:0/1444316704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100094b0 con 0x7f88200ccbe0 2024-09-13T16:53:01.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881f400700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200b3e20 0x7f88201e25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:01.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f881fe00700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88201e4810 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f881800ca80 tx=0x7f881800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:01.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.227+0000 7f8817400700 1 -- 172.21.15.3:0/1444316704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818007480 con 0x7f88200ccbe0 2024-09-13T16:53:01.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.231+0000 7f8817400700 1 -- 172.21.15.3:0/1444316704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88180075e0 con 0x7f88200ccbe0 2024-09-13T16:53:01.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.231+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88201e5040 con 0x7f88200ccbe0 2024-09-13T16:53:01.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.231+0000 7f8817400700 1 -- 172.21.15.3:0/1444316704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818007950 con 0x7f88200ccbe0 2024-09-13T16:53:01.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.231+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88201e5600 con 0x7f88200ccbe0 2024-09-13T16:53:01.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.231+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88200cbd60 con 0x7f88200ccbe0 2024-09-13T16:53:01.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.231+0000 7f8817400700 1 -- 172.21.15.3:0/1444316704 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f881801b3b0 con 0x7f88200ccbe0 2024-09-13T16:53:01.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.235+0000 7f8817400700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8800066730 0x7f8800068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.235+0000 7f8817400700 1 -- 172.21.15.3:0/1444316704 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8818083b40 con 0x7f88200ccbe0 2024-09-13T16:53:01.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.235+0000 7f881f400700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8800066730 0x7f8800068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.235+0000 7f881f400700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8800066730 0x7f8800068bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f88100090b0 tx=0x7f8810008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:01.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.239+0000 7f8817400700 1 -- 172.21.15.3:0/1444316704 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8818050d80 con 0x7f88200ccbe0 2024-09-13T16:53:01.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:01.471+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f882006cec0 con 0x7f88200ccbe0 2024-09-13T16:53:02.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.135+0000 7f8817400700 1 -- 172.21.15.3:0/1444316704 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8818017020 con 0x7f88200ccbe0 2024-09-13T16:53:02.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8800066730 msgr2=0x7f8800068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8800066730 0x7f8800068bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f88100090b0 tx=0x7f8810008030 comp rx=0 tx=0).stop 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 msgr2=0x7f88201e4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88201e4810 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f881800ca80 tx=0x7f881800cb60 comp rx=0 tx=0).stop 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 shutdown_connections 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f8800066730 0x7f8800068bf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200ccbe0 0x7f88201e4810 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88200cc530 0x7f8820147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 --2- 172.21.15.3:0/1444316704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200b3e20 0x7f88201e25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 >> 172.21.15.3:0/1444316704 conn(0x7f8820071680 msgr2=0x7f88200b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 shutdown_connections 2024-09-13T16:53:02.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.139+0000 7f8826600700 1 -- 172.21.15.3:0/1444316704 wait complete. 2024-09-13T16:53:02.174 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T16:53:02.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T16:53:02.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.359+0000 7f14a7c00700 1 Processor -- start 2024-09-13T16:53:02.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.359+0000 7f14a7c00700 1 -- start start 2024-09-13T16:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a7c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f14a015af60 0x7f14a00ac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a7c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a006b780 con 0x7f14a006a390 2024-09-13T16:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a7c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a00724e0 con 0x7f14a00b2c40 2024-09-13T16:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a7c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a006bd90 con 0x7f14a015af60 2024-09-13T16:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a5c00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14a006a390 0x7f14a00cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44344/0 (socket says 172.21.15.3:44344) 2024-09-13T16:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a5c00700 1 -- 172.21.15.3:0/1184886836 learned_addr learned my addr 172.21.15.3:0/1184886836 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a5200700 1 --2- 172.21.15.3:0/1184886836 >> v2:172.21.15.157:3300/0 conn(0x7f14a015af60 0x7f14a00ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f149f400700 1 -- 172.21.15.3:0/1184886836 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551890989 0 0) 0x7f14a00724e0 con 0x7f14a00b2c40 2024-09-13T16:53:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f149f400700 1 -- 172.21.15.3:0/1184886836 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14880035d0 con 0x7f14a00b2c40 2024-09-13T16:53:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f149f400700 1 -- 172.21.15.3:0/1184886836 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521032472 0 0) 0x7f14a006b780 con 0x7f14a006a390 2024-09-13T16:53:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a5200700 1 -- 172.21.15.3:0/1184886836 >> v1:172.21.15.157:6789/0 conn(0x7f14a00b2c40 legacy=0x7f14a00ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a5200700 1 -- 172.21.15.3:0/1184886836 >> v1:172.21.15.3:6789/0 conn(0x7f14a006a390 legacy=0x7f14a00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a5200700 1 -- 172.21.15.3:0/1184886836 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a006ba20 con 0x7f14a015af60 2024-09-13T16:53:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f14a5200700 1 --2- 172.21.15.3:0/1184886836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a015af60 0x7f14a00ac030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1494009d90 tx=0x7f149400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b1444d09327ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f149f400700 1 -- 172.21.15.3:0/1184886836 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400e070 con 0x7f14a015af60 2024-09-13T16:53:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f149f400700 1 -- 172.21.15.3:0/1184886836 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149400bb20 con 0x7f14a015af60 2024-09-13T16:53:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.363+0000 7f149f400700 1 -- 172.21.15.3:0/1184886836 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400bed0 con 0x7f14a015af60 2024-09-13T16:53:02.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.367+0000 7f14a7c00700 1 -- 172.21.15.3:0/1184886836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a015af60 msgr2=0x7f14a00ac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.367+0000 7f14a7c00700 1 --2- 172.21.15.3:0/1184886836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a015af60 0x7f14a00ac030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1494009d90 tx=0x7f149400b330 comp rx=0 tx=0).stop 2024-09-13T16:53:02.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.367+0000 7f14a7c00700 1 -- 172.21.15.3:0/1184886836 shutdown_connections 2024-09-13T16:53:02.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.367+0000 7f14a7c00700 1 --2- 172.21.15.3:0/1184886836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a015af60 0x7f14a00ac030 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.367+0000 7f14a7c00700 1 -- 172.21.15.3:0/1184886836 >> 172.21.15.3:0/1184886836 conn(0x7f14a0070a10 msgr2=0x7f14a00b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:02.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.367+0000 7f14a7c00700 1 -- 172.21.15.3:0/1184886836 shutdown_connections 2024-09-13T16:53:02.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.367+0000 7f14a7c00700 1 -- 172.21.15.3:0/1184886836 wait complete. 2024-09-13T16:53:02.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a7c00700 1 Processor -- start 2024-09-13T16:53:02.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a7c00700 1 -- start start 2024-09-13T16:53:02.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a7c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14a006a390 0x7f14a014f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a7c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a00b2c40 0x7f14a015d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a7c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14a015af60 0x7f14a00718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a7c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01fa530 con 0x7f14a015af60 2024-09-13T16:53:02.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a7c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a015dfd0 con 0x7f14a00b2c40 2024-09-13T16:53:02.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a7c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01fa780 con 0x7f14a006a390 2024-09-13T16:53:02.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14a015af60 0x7f14a00718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14a015af60 0x7f14a00718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39890/0 (socket says 172.21.15.3:39890) 2024-09-13T16:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5c00700 1 -- 172.21.15.3:0/3712996605 learned_addr learned my addr 172.21.15.3:0/3712996605 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f149fe00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a00b2c40 0x7f14a015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5200700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14a006a390 0x7f14a014f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5c00700 1 -- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14a006a390 msgr2=0x7f14a014f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14a006a390 0x7f14a014f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5c00700 1 -- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a00b2c40 msgr2=0x7f14a015d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a00b2c40 0x7f14a015d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.371+0000 7f14a5c00700 1 -- 172.21.15.3:0/3712996605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494009a70 con 0x7f14a015af60 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.375+0000 7f14a5c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14a015af60 0x7f14a00718c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f149800c550 tx=0x7f149800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.375+0000 7f149cc00700 1 -- 172.21.15.3:0/3712996605 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498004ca0 con 0x7f14a015af60 2024-09-13T16:53:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.375+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a01faa60 con 0x7f14a015af60 2024-09-13T16:53:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.375+0000 7f149cc00700 1 -- 172.21.15.3:0/3712996605 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1498004e40 con 0x7f14a015af60 2024-09-13T16:53:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.375+0000 7f149cc00700 1 -- 172.21.15.3:0/3712996605 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498007310 con 0x7f14a015af60 2024-09-13T16:53:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.375+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a014f940 con 0x7f14a015af60 2024-09-13T16:53:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.375+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a00724e0 con 0x7f14a015af60 2024-09-13T16:53:02.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.379+0000 7f149cc00700 1 -- 172.21.15.3:0/3712996605 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1498007470 con 0x7f14a015af60 2024-09-13T16:53:02.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.379+0000 7f149cc00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1480066730 0x7f1480068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.379+0000 7f149cc00700 1 -- 172.21.15.3:0/3712996605 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1498083ee0 con 0x7f14a015af60 2024-09-13T16:53:02.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.379+0000 7f14a5200700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1480066730 0x7f1480068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.383+0000 7f14a5200700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1480066730 0x7f1480068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1494014040 tx=0x7f1494011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.387+0000 7f149cc00700 1 -- 172.21.15.3:0/3712996605 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1498051080 con 0x7f14a015af60 2024-09-13T16:53:02.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:02.607+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f14a01557b0 con 0x7f14a015af60 2024-09-13T16:53:03.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.619+0000 7f149cc00700 1 -- 172.21.15.3:0/3712996605 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1498007720 con 0x7f14a015af60 2024-09-13T16:53:03.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1480066730 msgr2=0x7f1480068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1480066730 0x7f1480068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1494014040 tx=0x7f1494011000 comp rx=0 tx=0).stop 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14a015af60 msgr2=0x7f14a00718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14a015af60 0x7f14a00718c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f149800c550 tx=0x7f149800c860 comp rx=0 tx=0).stop 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 shutdown_connections 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1480066730 0x7f1480068bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14a015af60 0x7f14a00718c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a00b2c40 0x7f14a015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 --2- 172.21.15.3:0/3712996605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14a006a390 0x7f14a014f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 >> 172.21.15.3:0/3712996605 conn(0x7f14a0070a10 msgr2=0x7f14a0159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 shutdown_connections 2024-09-13T16:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.623+0000 7f14a7c00700 1 -- 172.21.15.3:0/3712996605 wait complete. 2024-09-13T16:53:03.655 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T16:53:03.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.823+0000 7fac9d600700 1 Processor -- start 2024-09-13T16:53:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.823+0000 7fac9d600700 1 -- start start 2024-09-13T16:53:03.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.823+0000 7fac9d600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac98070db0 0x7fac98071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:03.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.823+0000 7fac9d600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9806c540 0x7fac9806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:03.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.823+0000 7fac9d600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980b3450 con 0x7fac9815b660 2024-09-13T16:53:03.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.823+0000 7fac9d600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980b40d0 con 0x7fac98070db0 2024-09-13T16:53:03.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac9d600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac98071780 con 0x7fac9806c540 2024-09-13T16:53:03.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac8fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac98070db0 0x7fac98071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:03.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac97400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9806c540 0x7fac9806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:03.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac8fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac98070db0 0x7fac98071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:51952/0 (socket says 172.21.15.3:51952) 2024-09-13T16:53:03.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac96a00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fac9815b660 0x7fac980b2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:54088/0 (socket says 172.21.15.3:54088) 2024-09-13T16:53:03.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac8fe00700 1 -- 172.21.15.3:0/4224841163 learned_addr learned my addr 172.21.15.3:0/4224841163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:03.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac8fe00700 1 -- 172.21.15.3:0/4224841163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9806c540 msgr2=0x7fac9806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:03.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac8fe00700 1 --2- 172.21.15.3:0/4224841163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9806c540 0x7fac9806c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac8fe00700 1 -- 172.21.15.3:0/4224841163 >> v1:172.21.15.157:6789/0 conn(0x7fac9815b660 legacy=0x7fac980b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:03.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac8fe00700 1 -- 172.21.15.3:0/4224841163 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980b3730 con 0x7fac98070db0 2024-09-13T16:53:03.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac8fe00700 1 --2- 172.21.15.3:0/4224841163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac98070db0 0x7fac98071240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fac80009910 tx=0x7fac80009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43bedc57bfedc3e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:03.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac96000700 1 -- 172.21.15.3:0/4224841163 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80011830 con 0x7fac98070db0 2024-09-13T16:53:03.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac96000700 1 -- 172.21.15.3:0/4224841163 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac80011990 con 0x7fac98070db0 2024-09-13T16:53:03.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac96000700 1 -- 172.21.15.3:0/4224841163 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80011d40 con 0x7fac98070db0 2024-09-13T16:53:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac9d600700 1 -- 172.21.15.3:0/4224841163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac98070db0 msgr2=0x7fac98071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.827+0000 7fac9d600700 1 --2- 172.21.15.3:0/4224841163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac98070db0 0x7fac98071240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fac80009910 tx=0x7fac80009c20 comp rx=0 tx=0).stop 2024-09-13T16:53:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.831+0000 7fac9d600700 1 -- 172.21.15.3:0/4224841163 shutdown_connections 2024-09-13T16:53:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.831+0000 7fac9d600700 1 --2- 172.21.15.3:0/4224841163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9806c540 0x7fac9806c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.831+0000 7fac9d600700 1 --2- 172.21.15.3:0/4224841163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac98070db0 0x7fac98071240 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.831+0000 7fac9d600700 1 -- 172.21.15.3:0/4224841163 >> 172.21.15.3:0/4224841163 conn(0x7fac9806a390 msgr2=0x7fac9806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.831+0000 7fac9d600700 1 -- 172.21.15.3:0/4224841163 shutdown_connections 2024-09-13T16:53:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.831+0000 7fac9d600700 1 -- 172.21.15.3:0/4224841163 wait complete. 2024-09-13T16:53:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac9d600700 1 Processor -- start 2024-09-13T16:53:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac9d600700 1 -- start start 2024-09-13T16:53:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac9d600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9806c540 0x7fac981f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac9d600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac98070db0 0x7fac98152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac9d600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9815b660 0x7fac98152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac9d600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9815a030 con 0x7fac98070db0 2024-09-13T16:53:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac9d600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98159440 con 0x7fac9806c540 2024-09-13T16:53:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac9d600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac98159690 con 0x7fac9815b660 2024-09-13T16:53:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac8fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac98070db0 0x7fac98152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac8fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac98070db0 0x7fac98152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47462/0 (socket says 172.21.15.3:47462) 2024-09-13T16:53:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac97400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9815b660 0x7fac98152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac8fe00700 1 -- 172.21.15.3:0/2510478414 learned_addr learned my addr 172.21.15.3:0/2510478414 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac96a00700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9806c540 0x7fac981f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac8fe00700 1 -- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9815b660 msgr2=0x7fac98152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac8fe00700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9815b660 0x7fac98152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac8fe00700 1 -- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9806c540 msgr2=0x7fac981f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac8fe00700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9806c540 0x7fac981f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.835+0000 7fac8fe00700 1 -- 172.21.15.3:0/2510478414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac80009570 con 0x7fac98070db0 2024-09-13T16:53:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.839+0000 7fac8fe00700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac98070db0 0x7fac98152040 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fac80000c00 tx=0x7fac80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.839+0000 7fac8ea00700 1 -- 172.21.15.3:0/2510478414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80004510 con 0x7fac98070db0 2024-09-13T16:53:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.839+0000 7fac8ea00700 1 -- 172.21.15.3:0/2510478414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac800036c0 con 0x7fac98070db0 2024-09-13T16:53:03.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.839+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac98159970 con 0x7fac98070db0 2024-09-13T16:53:03.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.839+0000 7fac8ea00700 1 -- 172.21.15.3:0/2510478414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80003a70 con 0x7fac98070db0 2024-09-13T16:53:03.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.839+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac981f6960 con 0x7fac98070db0 2024-09-13T16:53:03.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.839+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9815ba40 con 0x7fac98070db0 2024-09-13T16:53:03.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.843+0000 7fac8ea00700 1 -- 172.21.15.3:0/2510478414 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fac80003bd0 con 0x7fac98070db0 2024-09-13T16:53:03.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.843+0000 7fac8ea00700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fac74066730 0x7fac74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:03.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.843+0000 7fac8ea00700 1 -- 172.21.15.3:0/2510478414 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fac80021030 con 0x7fac98070db0 2024-09-13T16:53:03.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.843+0000 7fac96a00700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fac74066730 0x7fac74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:03.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.847+0000 7fac96a00700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fac74066730 0x7fac74068bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fac88000f60 tx=0x7fac88007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:03.851+0000 7fac8ea00700 1 -- 172.21.15.3:0/2510478414 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fac80052b10 con 0x7fac98070db0 2024-09-13T16:53:04.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.083+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac9806b350 con 0x7fac98070db0 2024-09-13T16:53:04.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.083+0000 7fac8ea00700 1 -- 172.21.15.3:0/2510478414 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4731 (secure 0 0 0) 0x7fac80015090 con 0x7fac98070db0 2024-09-13T16:53:04.089 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:04.090 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5327,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617784475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1617784475},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617784475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5812,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4227223541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4227223541},{"type":"v1","addr":"172.21.15.3:6833","nonce":4227223541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2335957931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2335957931},{"type":"v1","addr":"172.21.15.157:6836","nonce":2335957931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:03.611651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5832},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5832":{"gid":5832,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/291442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":291442393},{"type":"v1","addr":"172.21.15.3:6835","nonce":291442393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fac74066730 msgr2=0x7fac74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fac74066730 0x7fac74068bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fac88000f60 tx=0x7fac88007ef0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac98070db0 msgr2=0x7fac98152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac98070db0 0x7fac98152040 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fac80000c00 tx=0x7fac80004770 comp rx=0 tx=0).stop 2024-09-13T16:53:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 shutdown_connections 2024-09-13T16:53:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fac74066730 0x7fac74068bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9815b660 0x7fac98152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac98070db0 0x7fac98152040 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 --2- 172.21.15.3:0/2510478414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9806c540 0x7fac981f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 >> 172.21.15.3:0/2510478414 conn(0x7fac9806a390 msgr2=0x7fac980cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 shutdown_connections 2024-09-13T16:53:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.087+0000 7fac9d600700 1 -- 172.21.15.3:0/2510478414 wait complete. 2024-09-13T16:53:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 30 2024-09-13T16:53:04.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:53:04.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.283+0000 7f9edf200700 1 Processor -- start 2024-09-13T16:53:04.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.283+0000 7f9edf200700 1 -- start start 2024-09-13T16:53:04.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9edf200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ee0144790 0x7f9ee006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9edf200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ee006cb20 0x7f9ee015b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9edf200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee00b4100 con 0x7f9ee01443b0 2024-09-13T16:53:04.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9edf200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee006cfb0 con 0x7f9ee006cb20 2024-09-13T16:53:04.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9edf200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee00ce8e0 con 0x7f9ee0144790 2024-09-13T16:53:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9ede800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ee006cb20 0x7f9ee015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9ede800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ee006cb20 0x7f9ee015b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47474/0 (socket says 172.21.15.3:47474) 2024-09-13T16:53:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9edde00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9ee01443b0 0x7f9ee00b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:54104/0 (socket says 172.21.15.3:54104) 2024-09-13T16:53:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9ede800700 1 -- 172.21.15.3:0/3586628243 learned_addr learned my addr 172.21.15.3:0/3586628243 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9ed5400700 1 --2- 172.21.15.3:0/3586628243 >> v2:172.21.15.157:3300/0 conn(0x7f9ee0144790 0x7f9ee006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9ede800700 1 -- 172.21.15.3:0/3586628243 >> v2:172.21.15.157:3300/0 conn(0x7f9ee0144790 msgr2=0x7f9ee006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9ede800700 1 --2- 172.21.15.3:0/3586628243 >> v2:172.21.15.157:3300/0 conn(0x7f9ee0144790 0x7f9ee006c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9ede800700 1 -- 172.21.15.3:0/3586628243 >> v1:172.21.15.157:6789/0 conn(0x7f9ee01443b0 legacy=0x7f9ee00b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.287+0000 7f9ede800700 1 -- 172.21.15.3:0/3586628243 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee00ccd30 con 0x7f9ee006cb20 2024-09-13T16:53:04.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9ede800700 1 --2- 172.21.15.3:0/3586628243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee006cb20 0x7f9ee015b220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ee006fc60 tx=0x7f9ed800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f209d473026357 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edd400700 1 -- 172.21.15.3:0/3586628243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800e070 con 0x7f9ee006cb20 2024-09-13T16:53:04.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edd400700 1 -- 172.21.15.3:0/3586628243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed800b910 con 0x7f9ee006cb20 2024-09-13T16:53:04.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edd400700 1 -- 172.21.15.3:0/3586628243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800bcc0 con 0x7f9ee006cb20 2024-09-13T16:53:04.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edf200700 1 -- 172.21.15.3:0/3586628243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee006cb20 msgr2=0x7f9ee015b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edf200700 1 --2- 172.21.15.3:0/3586628243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee006cb20 0x7f9ee015b220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ee006fc60 tx=0x7f9ed800b330 comp rx=0 tx=0).stop 2024-09-13T16:53:04.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edf200700 1 -- 172.21.15.3:0/3586628243 shutdown_connections 2024-09-13T16:53:04.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edf200700 1 --2- 172.21.15.3:0/3586628243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee006cb20 0x7f9ee015b220 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edf200700 1 --2- 172.21.15.3:0/3586628243 >> v2:172.21.15.157:3300/0 conn(0x7f9ee0144790 0x7f9ee006c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edf200700 1 -- 172.21.15.3:0/3586628243 >> 172.21.15.3:0/3586628243 conn(0x7f9ee00b0cf0 msgr2=0x7f9ee00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:04.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edf200700 1 -- 172.21.15.3:0/3586628243 shutdown_connections 2024-09-13T16:53:04.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.291+0000 7f9edf200700 1 -- 172.21.15.3:0/3586628243 wait complete. 2024-09-13T16:53:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edf200700 1 Processor -- start 2024-09-13T16:53:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edf200700 1 -- start start 2024-09-13T16:53:04.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edf200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee006cb20 0x7f9ee013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edf200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee01443b0 0x7f9ee013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edf200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee013a7e0 0x7f9ee01e24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edf200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee013d090 con 0x7f9ee01443b0 2024-09-13T16:53:04.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edf200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01e5e30 con 0x7f9ee006cb20 2024-09-13T16:53:04.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edf200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01e5b50 con 0x7f9ee013a7e0 2024-09-13T16:53:04.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee01443b0 0x7f9ee013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee01443b0 0x7f9ee013d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47486/0 (socket says 172.21.15.3:47486) 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 -- 172.21.15.3:0/1791350091 learned_addr learned my addr 172.21.15.3:0/1791350091 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edde00700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee006cb20 0x7f9ee013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ede800700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee013a7e0 0x7f9ee01e24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 -- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee013a7e0 msgr2=0x7f9ee01e24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee013a7e0 0x7f9ee01e24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 -- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee006cb20 msgr2=0x7f9ee013ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee006cb20 0x7f9ee013ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 -- 172.21.15.3:0/1791350091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8009b30 con 0x7f9ee01443b0 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9edde00700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee006cb20 0x7f9ee013ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.295+0000 7f9ed5400700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee01443b0 0x7f9ee013d520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800e9d0 tx=0x7f9ec800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.299+0000 7f9ed6a00700 1 -- 172.21.15.3:0/1791350091 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8009720 con 0x7f9ee01443b0 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.299+0000 7f9ed6a00700 1 -- 172.21.15.3:0/1791350091 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec800c850 con 0x7f9ee01443b0 2024-09-13T16:53:04.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.299+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee013ca40 con 0x7f9ee01443b0 2024-09-13T16:53:04.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.299+0000 7f9ed6a00700 1 -- 172.21.15.3:0/1791350091 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800cbc0 con 0x7f9ee01443b0 2024-09-13T16:53:04.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.299+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee013b860 con 0x7f9ee01443b0 2024-09-13T16:53:04.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.299+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee013d240 con 0x7f9ee01443b0 2024-09-13T16:53:04.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.299+0000 7f9ed6a00700 1 -- 172.21.15.3:0/1791350091 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ec800cd20 con 0x7f9ee01443b0 2024-09-13T16:53:04.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.303+0000 7f9ed6a00700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ebc066760 0x7f9ebc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.303+0000 7f9ed6a00700 1 -- 172.21.15.3:0/1791350091 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9ec8084360 con 0x7f9ee01443b0 2024-09-13T16:53:04.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.303+0000 7f9edde00700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ebc066760 0x7f9ebc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.303+0000 7f9edde00700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ebc066760 0x7f9ebc068c20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000f60 tx=0x7f9ed0007e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.307+0000 7f9ed6a00700 1 -- 172.21.15.3:0/1791350091 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ee013d240 con 0x7f9ee01443b0 2024-09-13T16:53:04.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.495+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ee00cca20 con 0x7f9ee01443b0 2024-09-13T16:53:04.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.495+0000 7f9ed6a00700 1 -- 172.21.15.3:0/1791350091 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16726 (secure 0 0 0) 0x7f9ec8055010 con 0x7f9ee01443b0 2024-09-13T16:53:04.504 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:04.504 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":58,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:53:01.118956+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:58.182202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:59.098263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ebc066760 msgr2=0x7f9ebc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ebc066760 0x7f9ebc068c20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000f60 tx=0x7f9ed0007e10 comp rx=0 tx=0).stop 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee01443b0 msgr2=0x7f9ee013d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee01443b0 0x7f9ee013d520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800e9d0 tx=0x7f9ec800ed90 comp rx=0 tx=0).stop 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 shutdown_connections 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ebc066760 0x7f9ebc068c20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee013a7e0 0x7f9ee01e24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee01443b0 0x7f9ee013d520 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 --2- 172.21.15.3:0/1791350091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee006cb20 0x7f9ee013ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 >> 172.21.15.3:0/1791350091 conn(0x7f9ee00b0cf0 msgr2=0x7f9ee01429b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 shutdown_connections 2024-09-13T16:53:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.499+0000 7f9edf200700 1 -- 172.21.15.3:0/1791350091 wait complete. 2024-09-13T16:53:04.529 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T16:53:04.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:53:04.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.699+0000 7f03cec00700 1 Processor -- start 2024-09-13T16:53:04.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cec00700 1 -- start start 2024-09-13T16:53:04.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cec00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f03c80b3f40 0x7f03c80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c806f8e0 0x7f03c806fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cec00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c815e070 con 0x7f03c815d4d0 2024-09-13T16:53:04.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cec00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c815e2a0 con 0x7f03c80b3f40 2024-09-13T16:53:04.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8070270 con 0x7f03c806f8e0 2024-09-13T16:53:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03ce200700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f03c815d4d0 0x7f03c815d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48712/0 (socket says 172.21.15.3:48712) 2024-09-13T16:53:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03ce200700 1 -- 172.21.15.3:0/1053246827 learned_addr learned my addr 172.21.15.3:0/1053246827 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cd800700 1 --2- 172.21.15.3:0/1053246827 >> v2:172.21.15.3:3300/0 conn(0x7f03c80b3f40 0x7f03c80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cce00700 1 --2- 172.21.15.3:0/1053246827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c806f8e0 0x7f03c806fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03c7e00700 1 -- 172.21.15.3:0/1053246827 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42886549 0 0) 0x7f03c815e070 con 0x7f03c815d4d0 2024-09-13T16:53:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03c7e00700 1 -- 172.21.15.3:0/1053246827 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b00035d0 con 0x7f03c815d4d0 2024-09-13T16:53:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03c7e00700 1 -- 172.21.15.3:0/1053246827 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 368129104 0 0) 0x7f03b00035d0 con 0x7f03c815d4d0 2024-09-13T16:53:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cd800700 1 -- 172.21.15.3:0/1053246827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c806f8e0 msgr2=0x7f03c806fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cd800700 1 --2- 172.21.15.3:0/1053246827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c806f8e0 0x7f03c806fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cd800700 1 -- 172.21.15.3:0/1053246827 >> v1:172.21.15.3:6789/0 conn(0x7f03c815d4d0 legacy=0x7f03c815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.703+0000 7f03cd800700 1 -- 172.21.15.3:0/1053246827 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c81617e0 con 0x7f03c80b3f40 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cd800700 1 --2- 172.21.15.3:0/1053246827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c80b4320 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009850 tx=0x7f03bc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5314cc9728b5c477 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03c7e00700 1 -- 172.21.15.3:0/1053246827 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00019f0 con 0x7f03c80b3f40 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03c7e00700 1 -- 172.21.15.3:0/1053246827 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc004030 con 0x7f03c80b3f40 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03c7e00700 1 -- 172.21.15.3:0/1053246827 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0043e0 con 0x7f03c80b3f40 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cec00700 1 -- 172.21.15.3:0/1053246827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 msgr2=0x7f03c80b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cec00700 1 --2- 172.21.15.3:0/1053246827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c80b4320 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009850 tx=0x7f03bc009b60 comp rx=0 tx=0).stop 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cec00700 1 -- 172.21.15.3:0/1053246827 shutdown_connections 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cec00700 1 --2- 172.21.15.3:0/1053246827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c806f8e0 0x7f03c806fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cec00700 1 --2- 172.21.15.3:0/1053246827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c80b4320 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cec00700 1 -- 172.21.15.3:0/1053246827 >> 172.21.15.3:0/1053246827 conn(0x7f03c80b2910 msgr2=0x7f03c80b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cec00700 1 -- 172.21.15.3:0/1053246827 shutdown_connections 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.707+0000 7f03cec00700 1 -- 172.21.15.3:0/1053246827 wait complete. 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cec00700 1 Processor -- start 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cec00700 1 -- start start 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c806f8e0 0x7f03c8142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cec00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c81f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c815d4d0 0x7f03c8144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cec00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8140ca0 con 0x7f03c80b3f40 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c806b4b0 con 0x7f03c806f8e0 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8140e90 con 0x7f03c815d4d0 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c81f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c81f5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47500/0 (socket says 172.21.15.3:47500) 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cce00700 1 -- 172.21.15.3:0/2442375030 learned_addr learned my addr 172.21.15.3:0/2442375030 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:04.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cd800700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c806f8e0 0x7f03c8142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03ce200700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c815d4d0 0x7f03c8144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cce00700 1 -- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c815d4d0 msgr2=0x7f03c8144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cce00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c815d4d0 0x7f03c8144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cce00700 1 -- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c806f8e0 msgr2=0x7f03c8142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cce00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c806f8e0 0x7f03c8142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cce00700 1 -- 172.21.15.3:0/2442375030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc0094b0 con 0x7f03c80b3f40 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03cd800700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c806f8e0 0x7f03c8142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.711+0000 7f03ce200700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c815d4d0 0x7f03c8144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.715+0000 7f03cce00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c81f5c40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e840 tx=0x7f03b400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.715+0000 7f03c5600700 1 -- 172.21.15.3:0/2442375030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40095a0 con 0x7f03c80b3f40 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.715+0000 7f03c5600700 1 -- 172.21.15.3:0/2442375030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b400c690 con 0x7f03c80b3f40 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.715+0000 7f03c5600700 1 -- 172.21.15.3:0/2442375030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b400ca00 con 0x7f03c80b3f40 2024-09-13T16:53:04.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.715+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c81411d0 con 0x7f03c80b3f40 2024-09-13T16:53:04.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.715+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c8141a10 con 0x7f03c80b3f40 2024-09-13T16:53:04.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.715+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c806fd30 con 0x7f03c80b3f40 2024-09-13T16:53:04.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.715+0000 7f03c5600700 1 -- 172.21.15.3:0/2442375030 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03b400cb60 con 0x7f03c80b3f40 2024-09-13T16:53:04.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.719+0000 7f03c5600700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f03a8066730 0x7f03a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.719+0000 7f03c5600700 1 -- 172.21.15.3:0/2442375030 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f03b4084320 con 0x7f03c80b3f40 2024-09-13T16:53:04.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.719+0000 7f03cd800700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f03a8066730 0x7f03a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.719+0000 7f03cd800700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f03a8066730 0x7f03a8068bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0090b0 tx=0x7f03bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.723+0000 7f03c5600700 1 -- 172.21.15.3:0/2442375030 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f03b40514b0 con 0x7f03c80b3f40 2024-09-13T16:53:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.915+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03c8160fe0 con 0x7f03a8066730 2024-09-13T16:53:04.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.931+0000 7f03c5600700 1 -- 172.21.15.3:0/2442375030 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276116 (secure 0 0 0) 0x7f03bc004520 con 0x7f03a8066730 2024-09-13T16:53:04.945 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f03a8066730 msgr2=0x7f03a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f03a8066730 0x7f03a8068bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0090b0 tx=0x7f03bc008030 comp rx=0 tx=0).stop 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 msgr2=0x7f03c81f5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c81f5c40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e840 tx=0x7f03b400ec00 comp rx=0 tx=0).stop 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 shutdown_connections 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f03a8066730 0x7f03a8068bf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03c815d4d0 0x7f03c8144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c80b3f40 0x7f03c81f5c40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 --2- 172.21.15.3:0/2442375030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c806f8e0 0x7f03c8142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 >> 172.21.15.3:0/2442375030 conn(0x7f03c80b2910 msgr2=0x7f03c815aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:04.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 shutdown_connections 2024-09-13T16:53:04.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:04.939+0000 7f03cec00700 1 -- 172.21.15.3:0/2442375030 wait complete. 2024-09-13T16:53:04.951 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:53:04.971 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-13T16:53:03.250433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238812,"kb_used_data":6244,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735908,"statfs":{"total":773094113280,"available":772849569792,"internally_reserved":0,"allocated":6393856,"data_stored":3497293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.658567"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.599259+0000","last_change":"2024-09-13T16:52:58.599259+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.599259+0000","last_undegraded":"2024-09-13T16:52:58.599259+0000","last_fullsized":"2024-09-13T16:52:58.599259+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:26.248204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.600194+0000","last_change":"2024-09-13T16:52:58.600194+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.600194+0000","last_undegraded":"2024-09-13T16:52:58.600194+0000","last_fullsized":"2024-09-13T16:52:58.600194+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:08.253591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.599081+0000","last_change":"2024-09-13T16:52:58.599081+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.599081+0000","last_undegraded":"2024-09-13T16:52:58.599081+0000","last_fullsized":"2024-09-13T16:52:58.599081+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:43.047842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.600286+0000","last_change":"2024-09-13T16:52:58.600286+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.600286+0000","last_undegraded":"2024-09-13T16:52:58.600286+0000","last_fullsized":"2024-09-13T16:52:58.600286+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:04.626963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.599424+0000","last_change":"2024-09-13T16:52:58.599424+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.599424+0000","last_undegraded":"2024-09-13T16:52:58.599424+0000","last_fullsized":"2024-09-13T16:52:58.599424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.192365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.594818+0000","last_change":"2024-09-13T16:52:58.594818+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.594818+0000","last_undegraded":"2024-09-13T16:52:58.594818+0000","last_fullsized":"2024-09-13T16:52:58.594818+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:17.644287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.598444+0000","last_change":"2024-09-13T16:52:58.598444+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.598444+0000","last_undegraded":"2024-09-13T16:52:58.598444+0000","last_fullsized":"2024-09-13T16:52:58.598444+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:59.996309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.599753+0000","last_change":"2024-09-13T16:52:58.599753+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.599753+0000","last_undegraded":"2024-09-13T16:52:58.599753+0000","last_fullsized":"2024-09-13T16:52:58.599753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:05.717453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.595794+0000","last_change":"2024-09-13T16:52:58.595794+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.595794+0000","last_undegraded":"2024-09-13T16:52:58.595794+0000","last_fullsized":"2024-09-13T16:52:58.595794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:39.779398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.594934+0000","last_change":"2024-09-13T16:52:58.594934+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.594934+0000","last_undegraded":"2024-09-13T16:52:58.594934+0000","last_fullsized":"2024-09-13T16:52:58.594934+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:28.274921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.596334+0000","last_change":"2024-09-13T16:52:58.596334+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.596334+0000","last_undegraded":"2024-09-13T16:52:58.596334+0000","last_fullsized":"2024-09-13T16:52:58.596334+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:16.728954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.594564+0000","last_change":"2024-09-13T16:52:58.594564+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.594564+0000","last_undegraded":"2024-09-13T16:52:58.594564+0000","last_fullsized":"2024-09-13T16:52:58.594564+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:17.009105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.598747+0000","last_change":"2024-09-13T16:52:58.598747+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.598747+0000","last_undegraded":"2024-09-13T16:52:58.598747+0000","last_fullsized":"2024-09-13T16:52:58.598747+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:21.045490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.598491+0000","last_change":"2024-09-13T16:52:58.598491+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.598491+0000","last_undegraded":"2024-09-13T16:52:58.598491+0000","last_fullsized":"2024-09-13T16:52:58.598491+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:28.274921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.595797+0000","last_change":"2024-09-13T16:52:58.595797+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.595797+0000","last_undegraded":"2024-09-13T16:52:58.595797+0000","last_fullsized":"2024-09-13T16:52:58.595797+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:37:08.657514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.594708+0000","last_change":"2024-09-13T16:52:58.594708+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.594708+0000","last_undegraded":"2024-09-13T16:52:58.594708+0000","last_fullsized":"2024-09-13T16:52:58.594708+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:09.759932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.598861+0000","last_change":"2024-09-13T16:52:58.598861+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.598861+0000","last_undegraded":"2024-09-13T16:52:58.598861+0000","last_fullsized":"2024-09-13T16:52:58.598861+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:28.274921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.603028+0000","last_change":"2024-09-13T16:52:58.603028+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.603028+0000","last_undegraded":"2024-09-13T16:52:58.603028+0000","last_fullsized":"2024-09-13T16:52:58.603028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:43.247599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.598423+0000","last_change":"2024-09-13T16:52:58.598423+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.598423+0000","last_undegraded":"2024-09-13T16:52:58.598423+0000","last_fullsized":"2024-09-13T16:52:58.598423+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:08.253591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.598418+0000","last_change":"2024-09-13T16:52:58.598418+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.598418+0000","last_undegraded":"2024-09-13T16:52:58.598418+0000","last_fullsized":"2024-09-13T16:52:58.598418+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:21.045490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.599598+0000","last_change":"2024-09-13T16:52:58.599598+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.599598+0000","last_undegraded":"2024-09-13T16:52:58.599598+0000","last_fullsized":"2024-09-13T16:52:58.599598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:59.996309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.595346+0000","last_change":"2024-09-13T16:52:58.595346+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.595346+0000","last_undegraded":"2024-09-13T16:52:58.595346+0000","last_fullsized":"2024-09-13T16:52:58.595346+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:43:49.517407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.599417+0000","last_change":"2024-09-13T16:52:58.599417+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.599417+0000","last_undegraded":"2024-09-13T16:52:58.599417+0000","last_fullsized":"2024-09-13T16:52:58.599417+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:01.514929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.602588+0000","last_change":"2024-09-13T16:52:58.602588+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.602588+0000","last_undegraded":"2024-09-13T16:52:58.602588+0000","last_fullsized":"2024-09-13T16:52:58.602588+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:12.756104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.598441+0000","last_change":"2024-09-13T16:52:58.598441+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.598441+0000","last_undegraded":"2024-09-13T16:52:58.598441+0000","last_fullsized":"2024-09-13T16:52:58.598441+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:17.644287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.595641+0000","last_change":"2024-09-13T16:52:58.595641+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.595641+0000","last_undegraded":"2024-09-13T16:52:58.595641+0000","last_fullsized":"2024-09-13T16:52:58.595641+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:44:03.491220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.600326+0000","last_change":"2024-09-13T16:52:58.600326+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.600326+0000","last_undegraded":"2024-09-13T16:52:58.600326+0000","last_fullsized":"2024-09-13T16:52:58.600326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:53:15.695398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.594506+0000","last_change":"2024-09-13T16:52:58.594506+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.594506+0000","last_undegraded":"2024-09-13T16:52:58.594506+0000","last_fullsized":"2024-09-13T16:52:58.594506+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:15.588431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.594524+0000","last_change":"2024-09-13T16:52:58.594524+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.594524+0000","last_undegraded":"2024-09-13T16:52:58.594524+0000","last_fullsized":"2024-09-13T16:52:58.594524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:39.839472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.594924+0000","last_change":"2024-09-13T16:52:58.594924+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.594924+0000","last_undegraded":"2024-09-13T16:52:58.594924+0000","last_fullsized":"2024-09-13T16:52:58.594924+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:26.248204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.600033+0000","last_change":"2024-09-13T16:52:58.600033+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.600033+0000","last_undegraded":"2024-09-13T16:52:58.600033+0000","last_fullsized":"2024-09-13T16:52:58.600033+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:18.467039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:52:59.590588+0000","last_change":"2024-09-13T16:52:59.590588+0000","last_active":"2024-09-13T16:52:59.590588+0000","last_peered":"2024-09-13T16:52:59.590588+0000","last_clean":"2024-09-13T16:52:59.590588+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:59.590588+0000","last_undegraded":"2024-09-13T16:52:59.590588+0000","last_fullsized":"2024-09-13T16:52:59.590588+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-13T16:52:58.600537+0000","last_change":"2024-09-13T16:52:58.600537+0000","last_active":"2024-09-13T16:52:58.575636+0000","last_peered":"2024-09-13T16:52:58.575636+0000","last_clean":"2024-09-13T16:52:58.575636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:52:58.600537+0000","last_undegraded":"2024-09-13T16:52:58.600537+0000","last_fullsized":"2024-09-13T16:52:58.600537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:50.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T16:52:56.627853+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:52:56.627853+0000","last_peered":"2024-09-13T16:52:56.627853+0000","last_clean":"2024-09-13T16:52:56.627853+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:52:56.627853+0000","last_undegraded":"2024-09-13T16:52:56.627853+0000","last_fullsized":"2024-09-13T16:52:56.627853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1249280,"data_stored":882239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.1279999999999999,"15min":1.145},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.4399999999999999,"5min":1.585,"15min":1.585},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0960000000000001,"15min":1.087},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.206}]},{"osd":1,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.25,"15min":1.27},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.3979999999999999,"5min":1.712,"15min":1.712},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2889999999999999,"15min":1.2909999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6339999999999999,"5min":1.897,"15min":1.897},"last":1.696}]},{"osd":2,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.3640000000000001,"15min":1.3859999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.7090000000000001,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.294,"15min":1.3240000000000001},"min":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.544,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.468}]},{"osd":3,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.248,"15min":1.2669999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.48,"5min":2.161,"15min":2.161},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2909999999999999,"15min":1.2849999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.663,"5min":1.704,"15min":1.704},"last":1.6100000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.165,"15min":1.153},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.329,"15min":1.3360000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6140000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.6739999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.373,"15min":1.367},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.806},{"interface":"front","average":{"1min":1.353,"5min":1.347,"15min":1.3460000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.867}]},{"osd":6,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3680000000000001,"15min":1.367},"min":{"1min":1.111,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.6359999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.4159999999999999,"15min":1.4259999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.651,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.504}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":655360,"data_stored":291871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.1040000000000001,"15min":1.129},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.369,"5min":1.49,"15min":1.49},"last":1.105},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.264,"15min":1.3140000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5660000000000001,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":1.365}]},{"osd":1,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.2869999999999999,"15min":1.3300000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.304,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.194,"15min":1.212},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.52,"5min":1.623,"15min":1.623},"last":0.97499999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.2969999999999999,"15min":1.329},"min":{"1min":0.82099999999999995,"5min":0.745,"15min":0.745},"max":{"1min":1.6519999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.2729999999999999,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.4379999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.89500000000000002}]},{"osd":3,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3879999999999999,"15min":1.4099999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.8280000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.4319999999999999,"15min":1.462},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.506,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":0.93899999999999995}]},{"osd":4,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2709999999999999,"15min":1.282},"min":{"1min":0.746,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5780000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.298,"15min":1.3169999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6100000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.163}]},{"osd":5,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.91300000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5249999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.419,"15min":1.4430000000000001},"min":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6679999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.2410000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.514,"15min":1.5369999999999999},"min":{"1min":1.115,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6120000000000001,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.27},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4870000000000001,"15min":1.5209999999999999},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.587,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.2190000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":643072,"data_stored":285734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1779999999999999,"15min":1.1799999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.581,"5min":2.222,"15min":2.222},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.105,"15min":1.095},"min":{"1min":0.78100000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.4179999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.4339999999999999,"15min":1.4530000000000001},"min":{"1min":0.85499999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.6779999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3460000000000001,"15min":1.349},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.72,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.52}]},{"osd":2,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.26,"15min":1.2689999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8140000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.266},"min":{"1min":0.90000000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.4770000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.611}]},{"osd":3,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.407,"15min":1.4399999999999999},"min":{"1min":0.90200000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.823,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3280000000000001,"15min":1.3400000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.7250000000000001,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.381}]},{"osd":4,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.212,"15min":1.214},"min":{"1min":0.81899999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.655,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.343,"15min":1.3460000000000001},"min":{"1min":0.96199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.649,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.3200000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.3100000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.873,"15min":0.873},"max":{"1min":1.552,"5min":1.887,"15min":1.887},"last":1.238},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.282,"15min":1.274},"min":{"1min":0.94999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5800000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.542}]},{"osd":7,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.5,"15min":1.514},"min":{"1min":1.109,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7549999999999999,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.466,"15min":1.48},"min":{"1min":1.052,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.704,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.581}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":643072,"data_stored":285734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0820000000000001,"15min":1.1000000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.266,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1,"5min":1.2190000000000001,"15min":1.2549999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3169999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.91400000000000003}]},{"osd":1,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.28,"15min":1.3009999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.474,"5min":1.919,"15min":1.919},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.1950000000000001,"15min":1.206},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.52,"5min":1.99,"15min":1.99},"last":0.96499999999999997}]},{"osd":2,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3009999999999999,"15min":1.327},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4710000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2130000000000001,"15min":1.2350000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.391,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.0249999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.4219999999999999,"15min":1.454},"min":{"1min":1.0289999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5609999999999999,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.4570000000000001,"15min":1.492},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.5549999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.391}]},{"osd":5,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.3120000000000001,"15min":1.3520000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3129999999999999,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.595},{"interface":"front","average":{"1min":1.214,"5min":1.3720000000000001,"15min":1.3979999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.546,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.2749999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.4910000000000001,"15min":1.5289999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5940000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.556},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.5760000000000001,"15min":1.6259999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.546,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.5149999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.5269999999999999,"15min":1.5609999999999999},"min":{"1min":1.135,"5min":1.107,"15min":1.107},"max":{"1min":1.6040000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.6200000000000001,"15min":1.667},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.6140000000000001,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.7849999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":291871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.995,"15min":1.0169999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4990000000000001,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.97899999999999998,"15min":0.98899999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.2589999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.032,"15min":1.0549999999999999},"min":{"1min":0.66400000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.391,"5min":1.931,"15min":1.931},"last":1.091},{"interface":"front","average":{"1min":1.052,"5min":1.1759999999999999,"15min":1.196},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.464,"5min":1.849,"15min":1.849},"last":1.1910000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.3069999999999999,"15min":1.339},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.702,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.369,"15min":1.403},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.657,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.91800000000000004}]},{"osd":4,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.4259999999999999,"15min":1.4630000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.599,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.492,"15min":1.518},"min":{"1min":1.0980000000000001,"5min":0.88,"15min":0.88},"max":{"1min":1.972,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.528}]},{"osd":5,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.609,"15min":1.645},"min":{"1min":0.90800000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.0569999999999999,"5min":2.79,"15min":2.79},"last":1.464},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.5900000000000001,"15min":1.6359999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.71,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.365}]},{"osd":6,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.655,"15min":1.7},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.8520000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.579,"15min":1.6259999999999999},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.649,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.399}]},{"osd":7,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.649,"15min":1.6839999999999999},"min":{"1min":1.103,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.012,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.71,"15min":1.7509999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":2.1320000000000001,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.591}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":291871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.93799999999999994,"15min":0.95299999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.01,"5min":1.113,"15min":1.1299999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.427,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.82199999999999995}]},{"osd":1,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.94899999999999995,"15min":0.97099999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.351,"5min":1.494,"15min":1.494},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0660000000000001,"15min":1.083},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4079999999999999,"5min":2.028,"15min":2.028},"last":0.93000000000000005}]},{"osd":3,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.198,"15min":1.1879999999999999},"min":{"1min":0.80200000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6870000000000001,"5min":1.948,"15min":1.948},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.161,"15min":1.145},"min":{"1min":0.71099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.046}]},{"osd":4,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3220000000000001,"15min":1.3260000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.452,"15min":1.47},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.141}]},{"osd":5,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.5609999999999999,"15min":1.5880000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.6629999999999998,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.407,"15min":1.423},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3029999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.2070000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.548,"15min":1.5740000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.5820000000000001,"15min":1.6100000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.2629999999999999,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.512}]},{"osd":7,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.6439999999999999,"15min":1.6799999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.73,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.383},{"interface":"front","average":{"1min":1.472,"5min":1.6419999999999999,"15min":1.6699999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.496,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.359}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":285734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89200000000000002,"15min":0.91000000000000003},"min":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0389999999999999,"5min":1.804,"15min":1.804},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.99099999999999999,"15min":0.998},"min":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5700000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.88800000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.212,"5min":1.954,"15min":1.954},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0569999999999999,"15min":1.075},"min":{"1min":0.67200000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3300000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.169,"15min":1.167},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7729999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1519999999999999,"15min":1.1559999999999999},"min":{"1min":0.68600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.6950000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.296,"15min":1.3180000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.6439999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.369,"15min":1.381},"min":{"1min":0.94799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.6020000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.538}]},{"osd":5,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.5209999999999999,"15min":1.5349999999999999},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.3570000000000002,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.5089999999999999,"15min":1.534},"min":{"1min":1.04,"5min":1,"15min":1},"max":{"1min":1.7130000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.595}]},{"osd":6,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5309999999999999,"15min":1.5449999999999999},"min":{"1min":1.1040000000000001,"5min":1.042,"15min":1.042},"max":{"1min":2.508,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.171},{"interface":"front","average":{"1min":1.321,"5min":1.476,"15min":1.502},"min":{"1min":0.95699999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.712,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.4770000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.536,"15min":1.556},"min":{"1min":1.0389999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":1.7450000000000001,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.425},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.5920000000000001,"15min":1.609},"min":{"1min":1.206,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.6030000000000002,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.615}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1249280,"data_stored":882239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.93300000000000005,"15min":0.95599999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1120000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3899999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.97099999999999997}]},{"osd":2,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.044,"15min":1.0469999999999999},"min":{"1min":0.65500000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.47,"5min":1.482,"15min":1.482},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.042,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.609,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":1.141}]},{"osd":3,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.024,"15min":1.032},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5489999999999999,"5min":1.653,"15min":1.653},"last":0.997},{"interface":"front","average":{"1min":1.149,"5min":1.226,"15min":1.2390000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5920000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.097}]},{"osd":4,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.361,"15min":1.3959999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.5940000000000001,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3520000000000001,"15min":1.363},"min":{"1min":0.88800000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5309999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.3879999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4750000000000001,"15min":1.5069999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6799999999999999,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":1.3},{"interface":"front","average":{"1min":1.276,"5min":1.452,"15min":1.4810000000000001},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.7969999999999999,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.2809999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5289999999999999,"15min":1.556},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.734,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.5600000000000001,"15min":1.5940000000000001},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.633,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":1.343}]},{"osd":7,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.6000000000000001,"15min":1.6339999999999999},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":1.698,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.258},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.524,"15min":1.548},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":1.7789999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.3640000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:53:04.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:53:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.187+0000 7fb78f200700 1 Processor -- start 2024-09-13T16:53:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.187+0000 7fb78f200700 1 -- start start 2024-09-13T16:53:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78f200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb79006d540 0x7fb7900b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78f200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79006b770 con 0x7fb7900ce750 2024-09-13T16:53:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78f200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79006ba50 con 0x7fb7900cf240 2024-09-13T16:53:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78f200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79006bd30 con 0x7fb79006d540 2024-09-13T16:53:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78de00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb7900ce750 0x7fb7900ceb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48728/0 (socket says 172.21.15.3:48728) 2024-09-13T16:53:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78de00700 1 -- 172.21.15.3:0/747594286 learned_addr learned my addr 172.21.15.3:0/747594286 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78d400700 1 --2- 172.21.15.3:0/747594286 >> v2:172.21.15.3:3300/0 conn(0x7fb79006d540 0x7fb7900b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78ca00700 1 -- 172.21.15.3:0/747594286 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862633397 0 0) 0x7fb79006b770 con 0x7fb7900ce750 2024-09-13T16:53:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78ca00700 1 -- 172.21.15.3:0/747594286 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7740035d0 con 0x7fb7900ce750 2024-09-13T16:53:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78d400700 1 -- 172.21.15.3:0/747594286 >> v1:172.21.15.157:6789/0 conn(0x7fb7900cf240 legacy=0x7fb7900b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78d400700 1 -- 172.21.15.3:0/747594286 >> v1:172.21.15.3:6789/0 conn(0x7fb7900ce750 legacy=0x7fb7900ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78ca00700 1 -- 172.21.15.3:0/747594286 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2015363004 0 0) 0x7fb7740035d0 con 0x7fb7900ce750 2024-09-13T16:53:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78d400700 1 -- 172.21.15.3:0/747594286 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79006c010 con 0x7fb79006d540 2024-09-13T16:53:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.191+0000 7fb78d400700 1 --2- 172.21.15.3:0/747594286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb79006d540 0x7fb7900b3f10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009850 tx=0x7fb77c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d6b862926f7f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78ca00700 1 -- 172.21.15.3:0/747594286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780001670 con 0x7fb79006d540 2024-09-13T16:53:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78ca00700 1 -- 172.21.15.3:0/747594286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c004030 con 0x7fb79006d540 2024-09-13T16:53:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78ca00700 1 -- 172.21.15.3:0/747594286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0043c0 con 0x7fb79006d540 2024-09-13T16:53:05.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78f200700 1 -- 172.21.15.3:0/747594286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb79006d540 msgr2=0x7fb7900b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78f200700 1 --2- 172.21.15.3:0/747594286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb79006d540 0x7fb7900b3f10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009850 tx=0x7fb77c009b60 comp rx=0 tx=0).stop 2024-09-13T16:53:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78f200700 1 -- 172.21.15.3:0/747594286 shutdown_connections 2024-09-13T16:53:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78f200700 1 --2- 172.21.15.3:0/747594286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb79006d540 0x7fb7900b3f10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78f200700 1 -- 172.21.15.3:0/747594286 >> 172.21.15.3:0/747594286 conn(0x7fb7900710b0 msgr2=0x7fb7900714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78f200700 1 -- 172.21.15.3:0/747594286 shutdown_connections 2024-09-13T16:53:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.195+0000 7fb78f200700 1 -- 172.21.15.3:0/747594286 wait complete. 2024-09-13T16:53:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78f200700 1 Processor -- start 2024-09-13T16:53:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78f200700 1 -- start start 2024-09-13T16:53:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78f200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb79006d540 0x7fb7900a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78f200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7900ce750 0x7fb7901535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7900cf240 0x7fb790157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78f200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb790153b00 con 0x7fb7900ce750 2024-09-13T16:53:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb790157b50 con 0x7fb7900cf240 2024-09-13T16:53:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78f200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb790155e60 con 0x7fb79006d540 2024-09-13T16:53:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7900ce750 0x7fb7901535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7900ce750 0x7fb7901535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47510/0 (socket says 172.21.15.3:47510) 2024-09-13T16:53:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78d400700 1 -- 172.21.15.3:0/2425949491 learned_addr learned my addr 172.21.15.3:0/2425949491 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78e800700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7900cf240 0x7fb790157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.199+0000 7fb78de00700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb79006d540 0x7fb7900a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78d400700 1 -- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb79006d540 msgr2=0x7fb7900a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78d400700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb79006d540 0x7fb7900a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78d400700 1 -- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7900cf240 msgr2=0x7fb790157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78d400700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7900cf240 0x7fb790157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78d400700 1 -- 172.21.15.3:0/2425949491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c0094b0 con 0x7fb7900ce750 2024-09-13T16:53:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78d400700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7900ce750 0x7fb7901535c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb790072110 tx=0x7fb77c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb786000700 1 -- 172.21.15.3:0/2425949491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c011830 con 0x7fb7900ce750 2024-09-13T16:53:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79015cd20 con 0x7fb7900ce750 2024-09-13T16:53:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb786000700 1 -- 172.21.15.3:0/2425949491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c01f070 con 0x7fb7900ce750 2024-09-13T16:53:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb786000700 1 -- 172.21.15.3:0/2425949491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c011bd0 con 0x7fb7900ce750 2024-09-13T16:53:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79015d2e0 con 0x7fb7900ce750 2024-09-13T16:53:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.203+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79015d4d0 con 0x7fb7900ce750 2024-09-13T16:53:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.207+0000 7fb786000700 1 -- 172.21.15.3:0/2425949491 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb77c011d30 con 0x7fb7900ce750 2024-09-13T16:53:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.207+0000 7fb786000700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb770066730 0x7fb770068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.207+0000 7fb786000700 1 -- 172.21.15.3:0/2425949491 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb77c021070 con 0x7fb7900ce750 2024-09-13T16:53:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.207+0000 7fb78de00700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb770066730 0x7fb770068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.207+0000 7fb78de00700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb770066730 0x7fb770068bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb780001eb0 tx=0x7fb780001c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.211+0000 7fb786000700 1 -- 172.21.15.3:0/2425949491 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb77c052fb0 con 0x7fb7900ce750 2024-09-13T16:53:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.399+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb79016bb70 con 0x7fb770066730 2024-09-13T16:53:05.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.411+0000 7fb786000700 1 -- 172.21.15.3:0/2425949491 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286850 (secure 0 0 0) 0x7fb780001670 con 0x7fb770066730 2024-09-13T16:53:05.425 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb770066730 msgr2=0x7fb770068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb770066730 0x7fb770068bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb780001eb0 tx=0x7fb780001c60 comp rx=0 tx=0).stop 2024-09-13T16:53:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7900ce750 msgr2=0x7fb7901535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7900ce750 0x7fb7901535c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb790072110 tx=0x7fb77c004770 comp rx=0 tx=0).stop 2024-09-13T16:53:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 shutdown_connections 2024-09-13T16:53:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb770066730 0x7fb770068bf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7900cf240 0x7fb790157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7900ce750 0x7fb7901535c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.419+0000 7fb78f200700 1 --2- 172.21.15.3:0/2425949491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb79006d540 0x7fb7900a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.423+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 >> 172.21.15.3:0/2425949491 conn(0x7fb7900710b0 msgr2=0x7fb7900b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:05.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.423+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 shutdown_connections 2024-09-13T16:53:05.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.423+0000 7fb78f200700 1 -- 172.21.15.3:0/2425949491 wait complete. 2024-09-13T16:53:05.430 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:53:05.450 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-13T16:53:05.251119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239208,"kb_used_data":6632,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735512,"statfs":{"total":773094113280,"available":772849164288,"internally_reserved":0,"allocated":6791168,"data_stored":3682908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001971"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388483+0000","last_change":"2024-09-13T16:53:00.136318+0000","last_active":"2024-09-13T16:53:01.388483+0000","last_peered":"2024-09-13T16:53:01.388483+0000","last_clean":"2024-09-13T16:53:01.388483+0000","last_became_active":"2024-09-13T16:53:00.135774+0000","last_became_peered":"2024-09-13T16:53:00.135774+0000","last_unstale":"2024-09-13T16:53:01.388483+0000","last_undegraded":"2024-09-13T16:53:01.388483+0000","last_fullsized":"2024-09-13T16:53:01.388483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:20.645982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132835+0000","last_change":"2024-09-13T16:53:00.147567+0000","last_active":"2024-09-13T16:53:01.132835+0000","last_peered":"2024-09-13T16:53:01.132835+0000","last_clean":"2024-09-13T16:53:01.132835+0000","last_became_active":"2024-09-13T16:53:00.147309+0000","last_became_peered":"2024-09-13T16:53:00.147309+0000","last_unstale":"2024-09-13T16:53:01.132835+0000","last_undegraded":"2024-09-13T16:53:01.132835+0000","last_fullsized":"2024-09-13T16:53:01.132835+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:29.289873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389889+0000","last_change":"2024-09-13T16:53:00.159977+0000","last_active":"2024-09-13T16:53:01.389889+0000","last_peered":"2024-09-13T16:53:01.389889+0000","last_clean":"2024-09-13T16:53:01.389889+0000","last_became_active":"2024-09-13T16:53:00.159709+0000","last_became_peered":"2024-09-13T16:53:00.159709+0000","last_unstale":"2024-09-13T16:53:01.389889+0000","last_undegraded":"2024-09-13T16:53:01.389889+0000","last_fullsized":"2024-09-13T16:53:01.389889+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:08.059123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132574+0000","last_change":"2024-09-13T16:53:00.152094+0000","last_active":"2024-09-13T16:53:01.132574+0000","last_peered":"2024-09-13T16:53:01.132574+0000","last_clean":"2024-09-13T16:53:01.132574+0000","last_became_active":"2024-09-13T16:53:00.151832+0000","last_became_peered":"2024-09-13T16:53:00.151832+0000","last_unstale":"2024-09-13T16:53:01.132574+0000","last_undegraded":"2024-09-13T16:53:01.132574+0000","last_fullsized":"2024-09-13T16:53:01.132574+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:26.207317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388479+0000","last_change":"2024-09-13T16:53:00.144297+0000","last_active":"2024-09-13T16:53:01.388479+0000","last_peered":"2024-09-13T16:53:01.388479+0000","last_clean":"2024-09-13T16:53:01.388479+0000","last_became_active":"2024-09-13T16:53:00.143897+0000","last_became_peered":"2024-09-13T16:53:00.143897+0000","last_unstale":"2024-09-13T16:53:01.388479+0000","last_undegraded":"2024-09-13T16:53:01.388479+0000","last_fullsized":"2024-09-13T16:53:01.388479+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:19.481991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388432+0000","last_change":"2024-09-13T16:53:00.144663+0000","last_active":"2024-09-13T16:53:01.388432+0000","last_peered":"2024-09-13T16:53:01.388432+0000","last_clean":"2024-09-13T16:53:01.388432+0000","last_became_active":"2024-09-13T16:53:00.144425+0000","last_became_peered":"2024-09-13T16:53:00.144425+0000","last_unstale":"2024-09-13T16:53:01.388432+0000","last_undegraded":"2024-09-13T16:53:01.388432+0000","last_fullsized":"2024-09-13T16:53:01.388432+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:06.809419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389438+0000","last_change":"2024-09-13T16:53:00.160759+0000","last_active":"2024-09-13T16:53:01.389438+0000","last_peered":"2024-09-13T16:53:01.389438+0000","last_clean":"2024-09-13T16:53:01.389438+0000","last_became_active":"2024-09-13T16:53:00.159893+0000","last_became_peered":"2024-09-13T16:53:00.159893+0000","last_unstale":"2024-09-13T16:53:01.389438+0000","last_undegraded":"2024-09-13T16:53:01.389438+0000","last_fullsized":"2024-09-13T16:53:01.389438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:40.117360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.284474+0000","last_change":"2024-09-13T16:53:00.152149+0000","last_active":"2024-09-13T16:53:01.284474+0000","last_peered":"2024-09-13T16:53:01.284474+0000","last_clean":"2024-09-13T16:53:01.284474+0000","last_became_active":"2024-09-13T16:53:00.151584+0000","last_became_peered":"2024-09-13T16:53:00.151584+0000","last_unstale":"2024-09-13T16:53:01.284474+0000","last_undegraded":"2024-09-13T16:53:01.284474+0000","last_fullsized":"2024-09-13T16:53:01.284474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:36.414079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.285182+0000","last_change":"2024-09-13T16:53:00.161304+0000","last_active":"2024-09-13T16:53:01.285182+0000","last_peered":"2024-09-13T16:53:01.285182+0000","last_clean":"2024-09-13T16:53:01.285182+0000","last_became_active":"2024-09-13T16:53:00.161066+0000","last_became_peered":"2024-09-13T16:53:00.161066+0000","last_unstale":"2024-09-13T16:53:01.285182+0000","last_undegraded":"2024-09-13T16:53:01.285182+0000","last_fullsized":"2024-09-13T16:53:01.285182+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:49.597108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388478+0000","last_change":"2024-09-13T16:53:00.137622+0000","last_active":"2024-09-13T16:53:01.388478+0000","last_peered":"2024-09-13T16:53:01.388478+0000","last_clean":"2024-09-13T16:53:01.388478+0000","last_became_active":"2024-09-13T16:53:00.137208+0000","last_became_peered":"2024-09-13T16:53:00.137208+0000","last_unstale":"2024-09-13T16:53:01.388478+0000","last_undegraded":"2024-09-13T16:53:01.388478+0000","last_fullsized":"2024-09-13T16:53:01.388478+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:36.547504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.131976+0000","last_change":"2024-09-13T16:53:00.141402+0000","last_active":"2024-09-13T16:53:01.131976+0000","last_peered":"2024-09-13T16:53:01.131976+0000","last_clean":"2024-09-13T16:53:01.131976+0000","last_became_active":"2024-09-13T16:53:00.141100+0000","last_became_peered":"2024-09-13T16:53:00.141100+0000","last_unstale":"2024-09-13T16:53:01.131976+0000","last_undegraded":"2024-09-13T16:53:01.131976+0000","last_fullsized":"2024-09-13T16:53:01.131976+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:47.399024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.285053+0000","last_change":"2024-09-13T16:53:00.160993+0000","last_active":"2024-09-13T16:53:01.285053+0000","last_peered":"2024-09-13T16:53:01.285053+0000","last_clean":"2024-09-13T16:53:01.285053+0000","last_became_active":"2024-09-13T16:53:00.160381+0000","last_became_peered":"2024-09-13T16:53:00.160381+0000","last_unstale":"2024-09-13T16:53:01.285053+0000","last_undegraded":"2024-09-13T16:53:01.285053+0000","last_fullsized":"2024-09-13T16:53:01.285053+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:25.679040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388025+0000","last_change":"2024-09-13T16:53:00.139056+0000","last_active":"2024-09-13T16:53:01.388025+0000","last_peered":"2024-09-13T16:53:01.388025+0000","last_clean":"2024-09-13T16:53:01.388025+0000","last_became_active":"2024-09-13T16:53:00.134657+0000","last_became_peered":"2024-09-13T16:53:00.134657+0000","last_unstale":"2024-09-13T16:53:01.388025+0000","last_undegraded":"2024-09-13T16:53:01.388025+0000","last_fullsized":"2024-09-13T16:53:01.388025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:45:37.488627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389880+0000","last_change":"2024-09-13T16:53:00.159904+0000","last_active":"2024-09-13T16:53:01.389880+0000","last_peered":"2024-09-13T16:53:01.389880+0000","last_clean":"2024-09-13T16:53:01.389880+0000","last_became_active":"2024-09-13T16:53:00.159643+0000","last_became_peered":"2024-09-13T16:53:00.159643+0000","last_unstale":"2024-09-13T16:53:01.389880+0000","last_undegraded":"2024-09-13T16:53:01.389880+0000","last_fullsized":"2024-09-13T16:53:01.389880+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:12.540325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.131494+0000","last_change":"2024-09-13T16:53:00.154798+0000","last_active":"2024-09-13T16:53:01.131494+0000","last_peered":"2024-09-13T16:53:01.131494+0000","last_clean":"2024-09-13T16:53:01.131494+0000","last_became_active":"2024-09-13T16:53:00.154537+0000","last_became_peered":"2024-09-13T16:53:00.154537+0000","last_unstale":"2024-09-13T16:53:01.131494+0000","last_undegraded":"2024-09-13T16:53:01.131494+0000","last_fullsized":"2024-09-13T16:53:01.131494+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:06.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.387915+0000","last_change":"2024-09-13T16:53:00.134919+0000","last_active":"2024-09-13T16:53:01.387915+0000","last_peered":"2024-09-13T16:53:01.387915+0000","last_clean":"2024-09-13T16:53:01.387915+0000","last_became_active":"2024-09-13T16:53:00.134602+0000","last_became_peered":"2024-09-13T16:53:00.134602+0000","last_unstale":"2024-09-13T16:53:01.387915+0000","last_undegraded":"2024-09-13T16:53:01.387915+0000","last_fullsized":"2024-09-13T16:53:01.387915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:21:49.069151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.390510+0000","last_change":"2024-09-13T16:53:00.156107+0000","last_active":"2024-09-13T16:53:01.390510+0000","last_peered":"2024-09-13T16:53:01.390510+0000","last_clean":"2024-09-13T16:53:01.390510+0000","last_became_active":"2024-09-13T16:53:00.155872+0000","last_became_peered":"2024-09-13T16:53:00.155872+0000","last_unstale":"2024-09-13T16:53:01.390510+0000","last_undegraded":"2024-09-13T16:53:01.390510+0000","last_fullsized":"2024-09-13T16:53:01.390510+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:18.024057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132840+0000","last_change":"2024-09-13T16:53:00.150213+0000","last_active":"2024-09-13T16:53:01.132840+0000","last_peered":"2024-09-13T16:53:01.132840+0000","last_clean":"2024-09-13T16:53:01.132840+0000","last_became_active":"2024-09-13T16:53:00.149904+0000","last_became_peered":"2024-09-13T16:53:00.149904+0000","last_unstale":"2024-09-13T16:53:01.132840+0000","last_undegraded":"2024-09-13T16:53:01.132840+0000","last_fullsized":"2024-09-13T16:53:01.132840+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:22.060442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389991+0000","last_change":"2024-09-13T16:53:00.144850+0000","last_active":"2024-09-13T16:53:01.389991+0000","last_peered":"2024-09-13T16:53:01.389991+0000","last_clean":"2024-09-13T16:53:01.389991+0000","last_became_active":"2024-09-13T16:53:00.144595+0000","last_became_peered":"2024-09-13T16:53:00.144595+0000","last_unstale":"2024-09-13T16:53:01.389991+0000","last_undegraded":"2024-09-13T16:53:01.389991+0000","last_fullsized":"2024-09-13T16:53:01.389991+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:15.992311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132075+0000","last_change":"2024-09-13T16:53:00.155986+0000","last_active":"2024-09-13T16:53:01.132075+0000","last_peered":"2024-09-13T16:53:01.132075+0000","last_clean":"2024-09-13T16:53:01.132075+0000","last_became_active":"2024-09-13T16:53:00.155746+0000","last_became_peered":"2024-09-13T16:53:00.155746+0000","last_unstale":"2024-09-13T16:53:01.132075+0000","last_undegraded":"2024-09-13T16:53:01.132075+0000","last_fullsized":"2024-09-13T16:53:01.132075+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:45:37.488627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.131823+0000","last_change":"2024-09-13T16:53:00.147811+0000","last_active":"2024-09-13T16:53:01.131823+0000","last_peered":"2024-09-13T16:53:01.131823+0000","last_clean":"2024-09-13T16:53:01.131823+0000","last_became_active":"2024-09-13T16:53:00.147465+0000","last_became_peered":"2024-09-13T16:53:00.147465+0000","last_unstale":"2024-09-13T16:53:01.131823+0000","last_undegraded":"2024-09-13T16:53:01.131823+0000","last_fullsized":"2024-09-13T16:53:01.131823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:07.290408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.284483+0000","last_change":"2024-09-13T16:53:00.152442+0000","last_active":"2024-09-13T16:53:01.284483+0000","last_peered":"2024-09-13T16:53:01.284483+0000","last_clean":"2024-09-13T16:53:01.284483+0000","last_became_active":"2024-09-13T16:53:00.152248+0000","last_became_peered":"2024-09-13T16:53:00.152248+0000","last_unstale":"2024-09-13T16:53:01.284483+0000","last_undegraded":"2024-09-13T16:53:01.284483+0000","last_fullsized":"2024-09-13T16:53:01.284483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:28:13.609433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.131659+0000","last_change":"2024-09-13T16:53:00.140974+0000","last_active":"2024-09-13T16:53:01.131659+0000","last_peered":"2024-09-13T16:53:01.131659+0000","last_clean":"2024-09-13T16:53:01.131659+0000","last_became_active":"2024-09-13T16:53:00.140727+0000","last_became_peered":"2024-09-13T16:53:00.140727+0000","last_unstale":"2024-09-13T16:53:01.131659+0000","last_undegraded":"2024-09-13T16:53:01.131659+0000","last_fullsized":"2024-09-13T16:53:01.131659+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:54.029253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389552+0000","last_change":"2024-09-13T16:53:00.152482+0000","last_active":"2024-09-13T16:53:01.389552+0000","last_peered":"2024-09-13T16:53:01.389552+0000","last_clean":"2024-09-13T16:53:01.389552+0000","last_became_active":"2024-09-13T16:53:00.148934+0000","last_became_peered":"2024-09-13T16:53:00.148934+0000","last_unstale":"2024-09-13T16:53:01.389552+0000","last_undegraded":"2024-09-13T16:53:01.389552+0000","last_fullsized":"2024-09-13T16:53:01.389552+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:06.734070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.390703+0000","last_change":"2024-09-13T16:53:00.155714+0000","last_active":"2024-09-13T16:53:01.390703+0000","last_peered":"2024-09-13T16:53:01.390703+0000","last_clean":"2024-09-13T16:53:01.390703+0000","last_became_active":"2024-09-13T16:53:00.155485+0000","last_became_peered":"2024-09-13T16:53:00.155485+0000","last_unstale":"2024-09-13T16:53:01.390703+0000","last_undegraded":"2024-09-13T16:53:01.390703+0000","last_fullsized":"2024-09-13T16:53:01.390703+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:47.723342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.285045+0000","last_change":"2024-09-13T16:53:00.164383+0000","last_active":"2024-09-13T16:53:01.285045+0000","last_peered":"2024-09-13T16:53:01.285045+0000","last_clean":"2024-09-13T16:53:01.285045+0000","last_became_active":"2024-09-13T16:53:00.164148+0000","last_became_peered":"2024-09-13T16:53:00.164148+0000","last_unstale":"2024-09-13T16:53:01.285045+0000","last_undegraded":"2024-09-13T16:53:01.285045+0000","last_fullsized":"2024-09-13T16:53:01.285045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:23.157312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388196+0000","last_change":"2024-09-13T16:53:00.140051+0000","last_active":"2024-09-13T16:53:01.388196+0000","last_peered":"2024-09-13T16:53:01.388196+0000","last_clean":"2024-09-13T16:53:01.388196+0000","last_became_active":"2024-09-13T16:53:00.139833+0000","last_became_peered":"2024-09-13T16:53:00.139833+0000","last_unstale":"2024-09-13T16:53:01.388196+0000","last_undegraded":"2024-09-13T16:53:01.388196+0000","last_fullsized":"2024-09-13T16:53:01.388196+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:35.672499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132173+0000","last_change":"2024-09-13T16:53:00.137764+0000","last_active":"2024-09-13T16:53:01.132173+0000","last_peered":"2024-09-13T16:53:01.132173+0000","last_clean":"2024-09-13T16:53:01.132173+0000","last_became_active":"2024-09-13T16:53:00.137324+0000","last_became_peered":"2024-09-13T16:53:00.137324+0000","last_unstale":"2024-09-13T16:53:01.132173+0000","last_undegraded":"2024-09-13T16:53:01.132173+0000","last_fullsized":"2024-09-13T16:53:01.132173+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:21:49.069151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389652+0000","last_change":"2024-09-13T16:53:00.157481+0000","last_active":"2024-09-13T16:53:01.389652+0000","last_peered":"2024-09-13T16:53:01.389652+0000","last_clean":"2024-09-13T16:53:01.389652+0000","last_became_active":"2024-09-13T16:53:00.157254+0000","last_became_peered":"2024-09-13T16:53:00.157254+0000","last_unstale":"2024-09-13T16:53:01.389652+0000","last_undegraded":"2024-09-13T16:53:01.389652+0000","last_fullsized":"2024-09-13T16:53:01.389652+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:10.774884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132382+0000","last_change":"2024-09-13T16:53:00.151688+0000","last_active":"2024-09-13T16:53:01.132382+0000","last_peered":"2024-09-13T16:53:01.132382+0000","last_clean":"2024-09-13T16:53:01.132382+0000","last_became_active":"2024-09-13T16:53:00.151383+0000","last_became_peered":"2024-09-13T16:53:00.151383+0000","last_unstale":"2024-09-13T16:53:01.132382+0000","last_undegraded":"2024-09-13T16:53:01.132382+0000","last_fullsized":"2024-09-13T16:53:01.132382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:51.984586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389468+0000","last_change":"2024-09-13T16:53:00.160570+0000","last_active":"2024-09-13T16:53:01.389468+0000","last_peered":"2024-09-13T16:53:01.389468+0000","last_clean":"2024-09-13T16:53:01.389468+0000","last_became_active":"2024-09-13T16:53:00.159540+0000","last_became_peered":"2024-09-13T16:53:00.159540+0000","last_unstale":"2024-09-13T16:53:01.389468+0000","last_undegraded":"2024-09-13T16:53:01.389468+0000","last_fullsized":"2024-09-13T16:53:01.389468+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:00.917306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388929+0000","last_change":"2024-09-13T16:53:00.150486+0000","last_active":"2024-09-13T16:53:01.388929+0000","last_peered":"2024-09-13T16:53:01.388929+0000","last_clean":"2024-09-13T16:53:01.388929+0000","last_became_active":"2024-09-13T16:53:00.149808+0000","last_became_peered":"2024-09-13T16:53:00.149808+0000","last_unstale":"2024-09-13T16:53:01.388929+0000","last_undegraded":"2024-09-13T16:53:01.388929+0000","last_fullsized":"2024-09-13T16:53:01.388929+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:26.302698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.131617+0000","last_change":"2024-09-13T16:53:00.154798+0000","last_active":"2024-09-13T16:53:01.131617+0000","last_peered":"2024-09-13T16:53:01.131617+0000","last_clean":"2024-09-13T16:53:01.131617+0000","last_became_active":"2024-09-13T16:53:00.154013+0000","last_became_peered":"2024-09-13T16:53:00.154013+0000","last_unstale":"2024-09-13T16:53:01.131617+0000","last_undegraded":"2024-09-13T16:53:01.131617+0000","last_fullsized":"2024-09-13T16:53:01.131617+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:53:16.710350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"57'4","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.159567+0000","last_change":"2024-09-13T16:52:59.618501+0000","last_active":"2024-09-13T16:53:00.159567+0000","last_peered":"2024-09-13T16:53:00.159567+0000","last_clean":"2024-09-13T16:53:00.159567+0000","last_became_active":"2024-09-13T16:52:59.616990+0000","last_became_peered":"2024-09-13T16:52:59.616990+0000","last_unstale":"2024-09-13T16:53:00.159567+0000","last_undegraded":"2024-09-13T16:53:00.159567+0000","last_fullsized":"2024-09-13T16:53:00.159567+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:26.248204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388942+0000","last_change":"2024-09-13T16:53:00.140688+0000","last_active":"2024-09-13T16:53:01.388942+0000","last_peered":"2024-09-13T16:53:01.388942+0000","last_clean":"2024-09-13T16:53:01.388942+0000","last_became_active":"2024-09-13T16:53:00.140327+0000","last_became_peered":"2024-09-13T16:53:00.140327+0000","last_unstale":"2024-09-13T16:53:01.388942+0000","last_undegraded":"2024-09-13T16:53:01.388942+0000","last_fullsized":"2024-09-13T16:53:01.388942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:50.669982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.151217+0000","last_change":"2024-09-13T16:52:59.634205+0000","last_active":"2024-09-13T16:53:00.151217+0000","last_peered":"2024-09-13T16:53:00.151217+0000","last_clean":"2024-09-13T16:53:00.151217+0000","last_became_active":"2024-09-13T16:52:59.633850+0000","last_became_peered":"2024-09-13T16:52:59.633850+0000","last_unstale":"2024-09-13T16:53:00.151217+0000","last_undegraded":"2024-09-13T16:53:00.151217+0000","last_fullsized":"2024-09-13T16:53:00.151217+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:08.253591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132440+0000","last_change":"2024-09-13T16:53:00.154323+0000","last_active":"2024-09-13T16:53:01.132440+0000","last_peered":"2024-09-13T16:53:01.132440+0000","last_clean":"2024-09-13T16:53:01.132440+0000","last_became_active":"2024-09-13T16:53:00.154049+0000","last_became_peered":"2024-09-13T16:53:00.154049+0000","last_unstale":"2024-09-13T16:53:01.132440+0000","last_undegraded":"2024-09-13T16:53:01.132440+0000","last_fullsized":"2024-09-13T16:53:01.132440+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:21.398231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.156014+0000","last_change":"2024-09-13T16:52:59.622037+0000","last_active":"2024-09-13T16:53:00.156014+0000","last_peered":"2024-09-13T16:53:00.156014+0000","last_clean":"2024-09-13T16:53:00.156014+0000","last_became_active":"2024-09-13T16:52:59.621721+0000","last_became_peered":"2024-09-13T16:52:59.621721+0000","last_unstale":"2024-09-13T16:53:00.156014+0000","last_undegraded":"2024-09-13T16:53:00.156014+0000","last_fullsized":"2024-09-13T16:53:00.156014+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:43.047842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.284861+0000","last_change":"2024-09-13T16:53:00.163867+0000","last_active":"2024-09-13T16:53:01.284861+0000","last_peered":"2024-09-13T16:53:01.284861+0000","last_clean":"2024-09-13T16:53:01.284861+0000","last_became_active":"2024-09-13T16:53:00.163478+0000","last_became_peered":"2024-09-13T16:53:00.163478+0000","last_unstale":"2024-09-13T16:53:01.284861+0000","last_undegraded":"2024-09-13T16:53:01.284861+0000","last_fullsized":"2024-09-13T16:53:01.284861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:45.304734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.162862+0000","last_change":"2024-09-13T16:52:59.639457+0000","last_active":"2024-09-13T16:53:00.162862+0000","last_peered":"2024-09-13T16:53:00.162862+0000","last_clean":"2024-09-13T16:53:00.162862+0000","last_became_active":"2024-09-13T16:52:59.639086+0000","last_became_peered":"2024-09-13T16:52:59.639086+0000","last_unstale":"2024-09-13T16:53:00.162862+0000","last_undegraded":"2024-09-13T16:53:00.162862+0000","last_fullsized":"2024-09-13T16:53:00.162862+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:04.626963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388676+0000","last_change":"2024-09-13T16:53:00.142865+0000","last_active":"2024-09-13T16:53:01.388676+0000","last_peered":"2024-09-13T16:53:01.388676+0000","last_clean":"2024-09-13T16:53:01.388676+0000","last_became_active":"2024-09-13T16:53:00.142601+0000","last_became_peered":"2024-09-13T16:53:00.142601+0000","last_unstale":"2024-09-13T16:53:01.388676+0000","last_undegraded":"2024-09-13T16:53:01.388676+0000","last_fullsized":"2024-09-13T16:53:01.388676+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:20.645982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.162261+0000","last_change":"2024-09-13T16:52:59.634022+0000","last_active":"2024-09-13T16:53:00.162261+0000","last_peered":"2024-09-13T16:53:00.162261+0000","last_clean":"2024-09-13T16:53:00.162261+0000","last_became_active":"2024-09-13T16:52:59.632495+0000","last_became_peered":"2024-09-13T16:52:59.632495+0000","last_unstale":"2024-09-13T16:53:00.162261+0000","last_undegraded":"2024-09-13T16:53:00.162261+0000","last_fullsized":"2024-09-13T16:53:00.162261+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.192365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132909+0000","last_change":"2024-09-13T16:53:00.152080+0000","last_active":"2024-09-13T16:53:01.132909+0000","last_peered":"2024-09-13T16:53:01.132909+0000","last_clean":"2024-09-13T16:53:01.132909+0000","last_became_active":"2024-09-13T16:53:00.151493+0000","last_became_peered":"2024-09-13T16:53:00.151493+0000","last_unstale":"2024-09-13T16:53:01.132909+0000","last_undegraded":"2024-09-13T16:53:01.132909+0000","last_fullsized":"2024-09-13T16:53:01.132909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:02.529882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.157613+0000","last_change":"2024-09-13T16:52:59.622548+0000","last_active":"2024-09-13T16:53:00.157613+0000","last_peered":"2024-09-13T16:53:00.157613+0000","last_clean":"2024-09-13T16:53:00.157613+0000","last_became_active":"2024-09-13T16:52:59.622171+0000","last_became_peered":"2024-09-13T16:52:59.622171+0000","last_unstale":"2024-09-13T16:53:00.157613+0000","last_undegraded":"2024-09-13T16:53:00.157613+0000","last_fullsized":"2024-09-13T16:53:00.157613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:17.644287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.285223+0000","last_change":"2024-09-13T16:53:00.164166+0000","last_active":"2024-09-13T16:53:01.285223+0000","last_peered":"2024-09-13T16:53:01.285223+0000","last_clean":"2024-09-13T16:53:01.285223+0000","last_became_active":"2024-09-13T16:53:00.163912+0000","last_became_peered":"2024-09-13T16:53:00.163912+0000","last_unstale":"2024-09-13T16:53:01.285223+0000","last_undegraded":"2024-09-13T16:53:01.285223+0000","last_fullsized":"2024-09-13T16:53:01.285223+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:11.842830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.130284+0000","last_change":"2024-09-13T16:52:59.619047+0000","last_active":"2024-09-13T16:53:00.130284+0000","last_peered":"2024-09-13T16:53:00.130284+0000","last_clean":"2024-09-13T16:53:00.130284+0000","last_became_active":"2024-09-13T16:52:59.618782+0000","last_became_peered":"2024-09-13T16:52:59.618782+0000","last_unstale":"2024-09-13T16:53:00.130284+0000","last_undegraded":"2024-09-13T16:53:00.130284+0000","last_fullsized":"2024-09-13T16:53:00.130284+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:59.996309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.132766+0000","last_change":"2024-09-13T16:53:00.149095+0000","last_active":"2024-09-13T16:53:01.132766+0000","last_peered":"2024-09-13T16:53:01.132766+0000","last_clean":"2024-09-13T16:53:01.132766+0000","last_became_active":"2024-09-13T16:53:00.148826+0000","last_became_peered":"2024-09-13T16:53:00.148826+0000","last_unstale":"2024-09-13T16:53:01.132766+0000","last_undegraded":"2024-09-13T16:53:01.132766+0000","last_fullsized":"2024-09-13T16:53:01.132766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:19.481991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.137389+0000","last_change":"2024-09-13T16:52:59.635533+0000","last_active":"2024-09-13T16:53:00.137389+0000","last_peered":"2024-09-13T16:53:00.137389+0000","last_clean":"2024-09-13T16:53:00.137389+0000","last_became_active":"2024-09-13T16:52:59.632571+0000","last_became_peered":"2024-09-13T16:52:59.632571+0000","last_unstale":"2024-09-13T16:53:00.137389+0000","last_undegraded":"2024-09-13T16:53:00.137389+0000","last_fullsized":"2024-09-13T16:53:00.137389+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:05.717453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388097+0000","last_change":"2024-09-13T16:53:00.148171+0000","last_active":"2024-09-13T16:53:01.388097+0000","last_peered":"2024-09-13T16:53:01.388097+0000","last_clean":"2024-09-13T16:53:01.388097+0000","last_became_active":"2024-09-13T16:53:00.147955+0000","last_became_peered":"2024-09-13T16:53:00.147955+0000","last_unstale":"2024-09-13T16:53:01.388097+0000","last_undegraded":"2024-09-13T16:53:01.388097+0000","last_fullsized":"2024-09-13T16:53:01.388097+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:45:37.488627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.137723+0000","last_change":"2024-09-13T16:52:59.624156+0000","last_active":"2024-09-13T16:53:00.137723+0000","last_peered":"2024-09-13T16:53:00.137723+0000","last_clean":"2024-09-13T16:53:00.137723+0000","last_became_active":"2024-09-13T16:52:59.623803+0000","last_became_peered":"2024-09-13T16:52:59.623803+0000","last_unstale":"2024-09-13T16:53:00.137723+0000","last_undegraded":"2024-09-13T16:53:00.137723+0000","last_fullsized":"2024-09-13T16:53:00.137723+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:39.779398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388763+0000","last_change":"2024-09-13T16:53:00.135558+0000","last_active":"2024-09-13T16:53:01.388763+0000","last_peered":"2024-09-13T16:53:01.388763+0000","last_clean":"2024-09-13T16:53:01.388763+0000","last_became_active":"2024-09-13T16:53:00.133523+0000","last_became_peered":"2024-09-13T16:53:00.133523+0000","last_unstale":"2024-09-13T16:53:01.388763+0000","last_undegraded":"2024-09-13T16:53:01.388763+0000","last_fullsized":"2024-09-13T16:53:01.388763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:24.962738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.128280+0000","last_change":"2024-09-13T16:52:59.635255+0000","last_active":"2024-09-13T16:53:00.128280+0000","last_peered":"2024-09-13T16:53:00.128280+0000","last_clean":"2024-09-13T16:53:00.128280+0000","last_became_active":"2024-09-13T16:52:59.634922+0000","last_became_peered":"2024-09-13T16:52:59.634922+0000","last_unstale":"2024-09-13T16:53:00.128280+0000","last_undegraded":"2024-09-13T16:53:00.128280+0000","last_fullsized":"2024-09-13T16:53:00.128280+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:28.274921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.390023+0000","last_change":"2024-09-13T16:53:00.155089+0000","last_active":"2024-09-13T16:53:01.390023+0000","last_peered":"2024-09-13T16:53:01.390023+0000","last_clean":"2024-09-13T16:53:01.390023+0000","last_became_active":"2024-09-13T16:53:00.154828+0000","last_became_peered":"2024-09-13T16:53:00.154828+0000","last_unstale":"2024-09-13T16:53:01.390023+0000","last_undegraded":"2024-09-13T16:53:01.390023+0000","last_fullsized":"2024-09-13T16:53:01.390023+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:40.854424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"57'3","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.151939+0000","last_change":"2024-09-13T16:52:59.625222+0000","last_active":"2024-09-13T16:53:00.151939+0000","last_peered":"2024-09-13T16:53:00.151939+0000","last_clean":"2024-09-13T16:53:00.151939+0000","last_became_active":"2024-09-13T16:52:59.624918+0000","last_became_peered":"2024-09-13T16:52:59.624918+0000","last_unstale":"2024-09-13T16:53:00.151939+0000","last_undegraded":"2024-09-13T16:53:00.151939+0000","last_fullsized":"2024-09-13T16:53:00.151939+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:16.728954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388529+0000","last_change":"2024-09-13T16:53:00.144495+0000","last_active":"2024-09-13T16:53:01.388529+0000","last_peered":"2024-09-13T16:53:01.388529+0000","last_clean":"2024-09-13T16:53:01.388529+0000","last_became_active":"2024-09-13T16:53:00.144193+0000","last_became_peered":"2024-09-13T16:53:00.144193+0000","last_unstale":"2024-09-13T16:53:01.388529+0000","last_undegraded":"2024-09-13T16:53:01.388529+0000","last_fullsized":"2024-09-13T16:53:01.388529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:07.290408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"57'6","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.171228+0000","last_change":"2024-09-13T16:52:59.638493+0000","last_active":"2024-09-13T16:53:00.171228+0000","last_peered":"2024-09-13T16:53:00.171228+0000","last_clean":"2024-09-13T16:53:00.171228+0000","last_became_active":"2024-09-13T16:52:59.638219+0000","last_became_peered":"2024-09-13T16:52:59.638219+0000","last_unstale":"2024-09-13T16:53:00.171228+0000","last_undegraded":"2024-09-13T16:53:00.171228+0000","last_fullsized":"2024-09-13T16:53:00.171228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:17.009105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389271+0000","last_change":"2024-09-13T16:53:00.142227+0000","last_active":"2024-09-13T16:53:01.389271+0000","last_peered":"2024-09-13T16:53:01.389271+0000","last_clean":"2024-09-13T16:53:01.389271+0000","last_became_active":"2024-09-13T16:53:00.141966+0000","last_became_peered":"2024-09-13T16:53:00.141966+0000","last_unstale":"2024-09-13T16:53:01.389271+0000","last_undegraded":"2024-09-13T16:53:01.389271+0000","last_fullsized":"2024-09-13T16:53:01.389271+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:52.079539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.147000+0000","last_change":"2024-09-13T16:52:59.631837+0000","last_active":"2024-09-13T16:53:00.147000+0000","last_peered":"2024-09-13T16:53:00.147000+0000","last_clean":"2024-09-13T16:53:00.147000+0000","last_became_active":"2024-09-13T16:52:59.631585+0000","last_became_peered":"2024-09-13T16:52:59.631585+0000","last_unstale":"2024-09-13T16:53:00.147000+0000","last_undegraded":"2024-09-13T16:53:00.147000+0000","last_fullsized":"2024-09-13T16:53:00.147000+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:21.045490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.390089+0000","last_change":"2024-09-13T16:53:00.152822+0000","last_active":"2024-09-13T16:53:01.390089+0000","last_peered":"2024-09-13T16:53:01.390089+0000","last_clean":"2024-09-13T16:53:01.390089+0000","last_became_active":"2024-09-13T16:53:00.152684+0000","last_became_peered":"2024-09-13T16:53:00.152684+0000","last_unstale":"2024-09-13T16:53:01.390089+0000","last_undegraded":"2024-09-13T16:53:01.390089+0000","last_fullsized":"2024-09-13T16:53:01.390089+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:43:50.532359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.130276+0000","last_change":"2024-09-13T16:52:59.634012+0000","last_active":"2024-09-13T16:53:00.130276+0000","last_peered":"2024-09-13T16:53:00.130276+0000","last_clean":"2024-09-13T16:53:00.130276+0000","last_became_active":"2024-09-13T16:52:59.633703+0000","last_became_peered":"2024-09-13T16:52:59.633703+0000","last_unstale":"2024-09-13T16:53:00.130276+0000","last_undegraded":"2024-09-13T16:53:00.130276+0000","last_fullsized":"2024-09-13T16:53:00.130276+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:28.274921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.285113+0000","last_change":"2024-09-13T16:53:00.150667+0000","last_active":"2024-09-13T16:53:01.285113+0000","last_peered":"2024-09-13T16:53:01.285113+0000","last_clean":"2024-09-13T16:53:01.285113+0000","last_became_active":"2024-09-13T16:53:00.149915+0000","last_became_peered":"2024-09-13T16:53:00.149915+0000","last_unstale":"2024-09-13T16:53:01.285113+0000","last_undegraded":"2024-09-13T16:53:01.285113+0000","last_fullsized":"2024-09-13T16:53:01.285113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:50.566329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.136142+0000","last_change":"2024-09-13T16:52:59.625913+0000","last_active":"2024-09-13T16:53:00.136142+0000","last_peered":"2024-09-13T16:53:00.136142+0000","last_clean":"2024-09-13T16:53:00.136142+0000","last_became_active":"2024-09-13T16:52:59.625487+0000","last_became_peered":"2024-09-13T16:52:59.625487+0000","last_unstale":"2024-09-13T16:53:00.136142+0000","last_undegraded":"2024-09-13T16:53:00.136142+0000","last_fullsized":"2024-09-13T16:53:00.136142+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:37:08.657514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.390488+0000","last_change":"2024-09-13T16:53:00.156335+0000","last_active":"2024-09-13T16:53:01.390488+0000","last_peered":"2024-09-13T16:53:01.390488+0000","last_clean":"2024-09-13T16:53:01.390488+0000","last_became_active":"2024-09-13T16:53:00.156126+0000","last_became_peered":"2024-09-13T16:53:00.156126+0000","last_unstale":"2024-09-13T16:53:01.390488+0000","last_undegraded":"2024-09-13T16:53:01.390488+0000","last_fullsized":"2024-09-13T16:53:01.390488+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:23:38.447452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"57'8","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.165412+0000","last_change":"2024-09-13T16:52:59.637975+0000","last_active":"2024-09-13T16:53:00.165412+0000","last_peered":"2024-09-13T16:53:00.165412+0000","last_clean":"2024-09-13T16:53:00.165412+0000","last_became_active":"2024-09-13T16:52:59.637373+0000","last_became_peered":"2024-09-13T16:52:59.637373+0000","last_unstale":"2024-09-13T16:53:00.165412+0000","last_undegraded":"2024-09-13T16:53:00.165412+0000","last_fullsized":"2024-09-13T16:53:00.165412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:09.759932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.284509+0000","last_change":"2024-09-13T16:53:00.148510+0000","last_active":"2024-09-13T16:53:01.284509+0000","last_peered":"2024-09-13T16:53:01.284509+0000","last_clean":"2024-09-13T16:53:01.284509+0000","last_became_active":"2024-09-13T16:53:00.148194+0000","last_became_peered":"2024-09-13T16:53:00.148194+0000","last_unstale":"2024-09-13T16:53:01.284509+0000","last_undegraded":"2024-09-13T16:53:01.284509+0000","last_fullsized":"2024-09-13T16:53:01.284509+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:57.785161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.159177+0000","last_change":"2024-09-13T16:52:59.638378+0000","last_active":"2024-09-13T16:53:00.159177+0000","last_peered":"2024-09-13T16:53:00.159177+0000","last_clean":"2024-09-13T16:53:00.159177+0000","last_became_active":"2024-09-13T16:52:59.638111+0000","last_became_peered":"2024-09-13T16:52:59.638111+0000","last_unstale":"2024-09-13T16:53:00.159177+0000","last_undegraded":"2024-09-13T16:53:00.159177+0000","last_fullsized":"2024-09-13T16:53:00.159177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:28.274921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388054+0000","last_change":"2024-09-13T16:53:00.144009+0000","last_active":"2024-09-13T16:53:01.388054+0000","last_peered":"2024-09-13T16:53:01.388054+0000","last_clean":"2024-09-13T16:53:01.388054+0000","last_became_active":"2024-09-13T16:53:00.139744+0000","last_became_peered":"2024-09-13T16:53:00.139744+0000","last_unstale":"2024-09-13T16:53:01.388054+0000","last_undegraded":"2024-09-13T16:53:01.388054+0000","last_fullsized":"2024-09-13T16:53:01.388054+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:10.875757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.150157+0000","last_change":"2024-09-13T16:52:59.628116+0000","last_active":"2024-09-13T16:53:00.150157+0000","last_peered":"2024-09-13T16:53:00.150157+0000","last_clean":"2024-09-13T16:53:00.150157+0000","last_became_active":"2024-09-13T16:52:59.627860+0000","last_became_peered":"2024-09-13T16:52:59.627860+0000","last_unstale":"2024-09-13T16:53:00.150157+0000","last_undegraded":"2024-09-13T16:53:00.150157+0000","last_fullsized":"2024-09-13T16:53:00.150157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:43.247599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388374+0000","last_change":"2024-09-13T16:53:00.139695+0000","last_active":"2024-09-13T16:53:01.388374+0000","last_peered":"2024-09-13T16:53:01.388374+0000","last_clean":"2024-09-13T16:53:01.388374+0000","last_became_active":"2024-09-13T16:53:00.139445+0000","last_became_peered":"2024-09-13T16:53:00.139445+0000","last_unstale":"2024-09-13T16:53:01.388374+0000","last_undegraded":"2024-09-13T16:53:01.388374+0000","last_fullsized":"2024-09-13T16:53:01.388374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:05.641915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.153344+0000","last_change":"2024-09-13T16:52:59.621330+0000","last_active":"2024-09-13T16:53:00.153344+0000","last_peered":"2024-09-13T16:53:00.153344+0000","last_clean":"2024-09-13T16:53:00.153344+0000","last_became_active":"2024-09-13T16:52:59.621054+0000","last_became_peered":"2024-09-13T16:52:59.621054+0000","last_unstale":"2024-09-13T16:53:00.153344+0000","last_undegraded":"2024-09-13T16:53:00.153344+0000","last_fullsized":"2024-09-13T16:53:00.153344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:08.253591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388296+0000","last_change":"2024-09-13T16:53:00.138781+0000","last_active":"2024-09-13T16:53:01.388296+0000","last_peered":"2024-09-13T16:53:01.388296+0000","last_clean":"2024-09-13T16:53:01.388296+0000","last_became_active":"2024-09-13T16:53:00.138489+0000","last_became_peered":"2024-09-13T16:53:00.138489+0000","last_unstale":"2024-09-13T16:53:01.388296+0000","last_undegraded":"2024-09-13T16:53:01.388296+0000","last_fullsized":"2024-09-13T16:53:01.388296+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:21.398231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.131235+0000","last_change":"2024-09-13T16:52:59.639344+0000","last_active":"2024-09-13T16:53:00.131235+0000","last_peered":"2024-09-13T16:53:00.131235+0000","last_clean":"2024-09-13T16:53:00.131235+0000","last_became_active":"2024-09-13T16:52:59.638927+0000","last_became_peered":"2024-09-13T16:52:59.638927+0000","last_unstale":"2024-09-13T16:53:00.131235+0000","last_undegraded":"2024-09-13T16:53:00.131235+0000","last_fullsized":"2024-09-13T16:53:00.131235+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:21.045490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389363+0000","last_change":"2024-09-13T16:53:00.137042+0000","last_active":"2024-09-13T16:53:01.389363+0000","last_peered":"2024-09-13T16:53:01.389363+0000","last_clean":"2024-09-13T16:53:01.389363+0000","last_became_active":"2024-09-13T16:53:00.136774+0000","last_became_peered":"2024-09-13T16:53:00.136774+0000","last_unstale":"2024-09-13T16:53:01.389363+0000","last_undegraded":"2024-09-13T16:53:01.389363+0000","last_fullsized":"2024-09-13T16:53:01.389363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:10.875757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.152387+0000","last_change":"2024-09-13T16:52:59.622340+0000","last_active":"2024-09-13T16:53:00.152387+0000","last_peered":"2024-09-13T16:53:00.152387+0000","last_clean":"2024-09-13T16:53:00.152387+0000","last_became_active":"2024-09-13T16:52:59.622041+0000","last_became_peered":"2024-09-13T16:52:59.622041+0000","last_unstale":"2024-09-13T16:53:00.152387+0000","last_undegraded":"2024-09-13T16:53:00.152387+0000","last_fullsized":"2024-09-13T16:53:00.152387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:59.996309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.131934+0000","last_change":"2024-09-13T16:53:00.155131+0000","last_active":"2024-09-13T16:53:01.131934+0000","last_peered":"2024-09-13T16:53:01.131934+0000","last_clean":"2024-09-13T16:53:01.131934+0000","last_became_active":"2024-09-13T16:53:00.154895+0000","last_became_peered":"2024-09-13T16:53:00.154895+0000","last_unstale":"2024-09-13T16:53:01.131934+0000","last_undegraded":"2024-09-13T16:53:01.131934+0000","last_fullsized":"2024-09-13T16:53:01.131934+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:20.645982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.140609+0000","last_change":"2024-09-13T16:52:59.641375+0000","last_active":"2024-09-13T16:53:00.140609+0000","last_peered":"2024-09-13T16:53:00.140609+0000","last_clean":"2024-09-13T16:53:00.140609+0000","last_became_active":"2024-09-13T16:52:59.640597+0000","last_became_peered":"2024-09-13T16:52:59.640597+0000","last_unstale":"2024-09-13T16:53:00.140609+0000","last_undegraded":"2024-09-13T16:53:00.140609+0000","last_fullsized":"2024-09-13T16:53:00.140609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:43:49.517407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.390303+0000","last_change":"2024-09-13T16:53:00.157774+0000","last_active":"2024-09-13T16:53:01.390303+0000","last_peered":"2024-09-13T16:53:01.390303+0000","last_clean":"2024-09-13T16:53:01.390303+0000","last_became_active":"2024-09-13T16:53:00.155063+0000","last_became_peered":"2024-09-13T16:53:00.155063+0000","last_unstale":"2024-09-13T16:53:01.390303+0000","last_undegraded":"2024-09-13T16:53:01.390303+0000","last_fullsized":"2024-09-13T16:53:01.390303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:16.603383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.137631+0000","last_change":"2024-09-13T16:52:59.628236+0000","last_active":"2024-09-13T16:53:00.137631+0000","last_peered":"2024-09-13T16:53:00.137631+0000","last_clean":"2024-09-13T16:53:00.137631+0000","last_became_active":"2024-09-13T16:52:59.627906+0000","last_became_peered":"2024-09-13T16:52:59.627906+0000","last_unstale":"2024-09-13T16:53:00.137631+0000","last_undegraded":"2024-09-13T16:53:00.137631+0000","last_fullsized":"2024-09-13T16:53:00.137631+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:01.514929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388701+0000","last_change":"2024-09-13T16:53:00.150772+0000","last_active":"2024-09-13T16:53:01.388701+0000","last_peered":"2024-09-13T16:53:01.388701+0000","last_clean":"2024-09-13T16:53:01.388701+0000","last_became_active":"2024-09-13T16:53:00.150434+0000","last_became_peered":"2024-09-13T16:53:00.150434+0000","last_unstale":"2024-09-13T16:53:01.388701+0000","last_undegraded":"2024-09-13T16:53:01.388701+0000","last_fullsized":"2024-09-13T16:53:01.388701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:52.079539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.133349+0000","last_change":"2024-09-13T16:52:59.628018+0000","last_active":"2024-09-13T16:53:00.133349+0000","last_peered":"2024-09-13T16:53:00.133349+0000","last_clean":"2024-09-13T16:53:00.133349+0000","last_became_active":"2024-09-13T16:52:59.626793+0000","last_became_peered":"2024-09-13T16:52:59.626793+0000","last_unstale":"2024-09-13T16:53:00.133349+0000","last_undegraded":"2024-09-13T16:53:00.133349+0000","last_fullsized":"2024-09-13T16:53:00.133349+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:12.756104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389184+0000","last_change":"2024-09-13T16:53:00.150665+0000","last_active":"2024-09-13T16:53:01.389184+0000","last_peered":"2024-09-13T16:53:01.389184+0000","last_clean":"2024-09-13T16:53:01.389184+0000","last_became_active":"2024-09-13T16:53:00.150152+0000","last_became_peered":"2024-09-13T16:53:00.150152+0000","last_unstale":"2024-09-13T16:53:01.389184+0000","last_undegraded":"2024-09-13T16:53:01.389184+0000","last_fullsized":"2024-09-13T16:53:01.389184+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:15.992311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.162739+0000","last_change":"2024-09-13T16:52:59.634013+0000","last_active":"2024-09-13T16:53:00.162739+0000","last_peered":"2024-09-13T16:53:00.162739+0000","last_clean":"2024-09-13T16:53:00.162739+0000","last_became_active":"2024-09-13T16:52:59.633733+0000","last_became_peered":"2024-09-13T16:52:59.633733+0000","last_unstale":"2024-09-13T16:53:00.162739+0000","last_undegraded":"2024-09-13T16:53:00.162739+0000","last_fullsized":"2024-09-13T16:53:00.162739+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:17.644287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.388662+0000","last_change":"2024-09-13T16:53:00.144860+0000","last_active":"2024-09-13T16:53:01.388662+0000","last_peered":"2024-09-13T16:53:01.388662+0000","last_clean":"2024-09-13T16:53:01.388662+0000","last_became_active":"2024-09-13T16:53:00.144586+0000","last_became_peered":"2024-09-13T16:53:00.144586+0000","last_unstale":"2024-09-13T16:53:01.388662+0000","last_undegraded":"2024-09-13T16:53:01.388662+0000","last_fullsized":"2024-09-13T16:53:01.388662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:45.754398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.136871+0000","last_change":"2024-09-13T16:52:59.631984+0000","last_active":"2024-09-13T16:53:00.136871+0000","last_peered":"2024-09-13T16:53:00.136871+0000","last_clean":"2024-09-13T16:53:00.136871+0000","last_became_active":"2024-09-13T16:52:59.631691+0000","last_became_peered":"2024-09-13T16:52:59.631691+0000","last_unstale":"2024-09-13T16:53:00.136871+0000","last_undegraded":"2024-09-13T16:53:00.136871+0000","last_fullsized":"2024-09-13T16:53:00.136871+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:44:03.491220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.284720+0000","last_change":"2024-09-13T16:53:00.148805+0000","last_active":"2024-09-13T16:53:01.284720+0000","last_peered":"2024-09-13T16:53:01.284720+0000","last_clean":"2024-09-13T16:53:01.284720+0000","last_became_active":"2024-09-13T16:53:00.148504+0000","last_became_peered":"2024-09-13T16:53:00.148504+0000","last_unstale":"2024-09-13T16:53:01.284720+0000","last_undegraded":"2024-09-13T16:53:01.284720+0000","last_fullsized":"2024-09-13T16:53:01.284720+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:25.128890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.137508+0000","last_change":"2024-09-13T16:52:59.634958+0000","last_active":"2024-09-13T16:53:00.137508+0000","last_peered":"2024-09-13T16:53:00.137508+0000","last_clean":"2024-09-13T16:53:00.137508+0000","last_became_active":"2024-09-13T16:52:59.631995+0000","last_became_peered":"2024-09-13T16:52:59.631995+0000","last_unstale":"2024-09-13T16:53:00.137508+0000","last_undegraded":"2024-09-13T16:53:00.137508+0000","last_fullsized":"2024-09-13T16:53:00.137508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:53:15.695398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389820+0000","last_change":"2024-09-13T16:53:00.159498+0000","last_active":"2024-09-13T16:53:01.389820+0000","last_peered":"2024-09-13T16:53:01.389820+0000","last_clean":"2024-09-13T16:53:01.389820+0000","last_became_active":"2024-09-13T16:53:00.159262+0000","last_became_peered":"2024-09-13T16:53:00.159262+0000","last_unstale":"2024-09-13T16:53:01.389820+0000","last_undegraded":"2024-09-13T16:53:01.389820+0000","last_fullsized":"2024-09-13T16:53:01.389820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:09.601011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.143733+0000","last_change":"2024-09-13T16:52:59.637798+0000","last_active":"2024-09-13T16:53:00.143733+0000","last_peered":"2024-09-13T16:53:00.143733+0000","last_clean":"2024-09-13T16:53:00.143733+0000","last_became_active":"2024-09-13T16:52:59.637469+0000","last_became_peered":"2024-09-13T16:52:59.637469+0000","last_unstale":"2024-09-13T16:53:00.143733+0000","last_undegraded":"2024-09-13T16:53:00.143733+0000","last_fullsized":"2024-09-13T16:53:00.143733+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:15.588431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.389062+0000","last_change":"2024-09-13T16:53:00.146732+0000","last_active":"2024-09-13T16:53:01.389062+0000","last_peered":"2024-09-13T16:53:01.389062+0000","last_clean":"2024-09-13T16:53:01.389062+0000","last_became_active":"2024-09-13T16:53:00.142706+0000","last_became_peered":"2024-09-13T16:53:00.142706+0000","last_unstale":"2024-09-13T16:53:01.389062+0000","last_undegraded":"2024-09-13T16:53:01.389062+0000","last_fullsized":"2024-09-13T16:53:01.389062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:45.754398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.140701+0000","last_change":"2024-09-13T16:52:59.639561+0000","last_active":"2024-09-13T16:53:00.140701+0000","last_peered":"2024-09-13T16:53:00.140701+0000","last_clean":"2024-09-13T16:53:00.140701+0000","last_became_active":"2024-09-13T16:52:59.637539+0000","last_became_peered":"2024-09-13T16:52:59.637539+0000","last_unstale":"2024-09-13T16:53:00.140701+0000","last_undegraded":"2024-09-13T16:53:00.140701+0000","last_fullsized":"2024-09-13T16:53:00.140701+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:39.839472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.390206+0000","last_change":"2024-09-13T16:53:00.160483+0000","last_active":"2024-09-13T16:53:01.390206+0000","last_peered":"2024-09-13T16:53:01.390206+0000","last_clean":"2024-09-13T16:53:01.390206+0000","last_became_active":"2024-09-13T16:53:00.159997+0000","last_became_peered":"2024-09-13T16:53:00.159997+0000","last_unstale":"2024-09-13T16:53:01.390206+0000","last_undegraded":"2024-09-13T16:53:01.390206+0000","last_fullsized":"2024-09-13T16:53:01.390206+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:29.660087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.126979+0000","last_change":"2024-09-13T16:52:59.630820+0000","last_active":"2024-09-13T16:53:00.126979+0000","last_peered":"2024-09-13T16:53:00.126979+0000","last_clean":"2024-09-13T16:53:00.126979+0000","last_became_active":"2024-09-13T16:52:59.630515+0000","last_became_peered":"2024-09-13T16:52:59.630515+0000","last_unstale":"2024-09-13T16:53:00.126979+0000","last_undegraded":"2024-09-13T16:53:00.126979+0000","last_fullsized":"2024-09-13T16:53:00.126979+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:26.248204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.285414+0000","last_change":"2024-09-13T16:53:00.163891+0000","last_active":"2024-09-13T16:53:01.285414+0000","last_peered":"2024-09-13T16:53:01.285414+0000","last_clean":"2024-09-13T16:53:01.285414+0000","last_became_active":"2024-09-13T16:53:00.163445+0000","last_became_peered":"2024-09-13T16:53:00.163445+0000","last_unstale":"2024-09-13T16:53:01.285414+0000","last_undegraded":"2024-09-13T16:53:01.285414+0000","last_fullsized":"2024-09-13T16:53:01.285414+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:28.903011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.127549+0000","last_change":"2024-09-13T16:52:59.623539+0000","last_active":"2024-09-13T16:53:00.127549+0000","last_peered":"2024-09-13T16:53:00.127549+0000","last_clean":"2024-09-13T16:53:00.127549+0000","last_became_active":"2024-09-13T16:52:59.623244+0000","last_became_peered":"2024-09-13T16:52:59.623244+0000","last_unstale":"2024-09-13T16:53:00.127549+0000","last_undegraded":"2024-09-13T16:53:00.127549+0000","last_fullsized":"2024-09-13T16:53:00.127549+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:18.467039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T16:53:01.131802+0000","last_change":"2024-09-13T16:53:00.134525+0000","last_active":"2024-09-13T16:53:01.131802+0000","last_peered":"2024-09-13T16:53:01.131802+0000","last_clean":"2024-09-13T16:53:01.131802+0000","last_became_active":"2024-09-13T16:53:00.134068+0000","last_became_peered":"2024-09-13T16:53:00.134068+0000","last_unstale":"2024-09-13T16:53:01.131802+0000","last_undegraded":"2024-09-13T16:53:01.131802+0000","last_fullsized":"2024-09-13T16:53:01.131802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:59.590588+0000","last_clean_scrub_stamp":"2024-09-13T16:52:59.590588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:36.547504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.138359+0000","last_change":"2024-09-13T16:52:59.633665+0000","last_active":"2024-09-13T16:53:00.138359+0000","last_peered":"2024-09-13T16:53:00.138359+0000","last_clean":"2024-09-13T16:53:00.138359+0000","last_became_active":"2024-09-13T16:52:59.633394+0000","last_became_peered":"2024-09-13T16:52:59.633394+0000","last_unstale":"2024-09-13T16:53:00.138359+0000","last_undegraded":"2024-09-13T16:53:00.138359+0000","last_fullsized":"2024-09-13T16:53:00.138359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:52:58.575636+0000","last_clean_scrub_stamp":"2024-09-13T16:52:58.575636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:50.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T16:53:00.135520+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:53:00.135520+0000","last_peered":"2024-09-13T16:53:00.135520+0000","last_clean":"2024-09-13T16:53:00.135520+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:53:00.135520+0000","last_undegraded":"2024-09-13T16:53:00.135520+0000","last_fullsized":"2024-09-13T16:53:00.135520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1294336,"data_stored":902673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.1279999999999999,"15min":1.145},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.4399999999999999,"5min":1.585,"15min":1.585},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0960000000000001,"15min":1.087},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.117}]},{"osd":1,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.25,"15min":1.27},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.3979999999999999,"5min":1.712,"15min":1.712},"last":1.151},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2889999999999999,"15min":1.2909999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6339999999999999,"5min":1.897,"15min":1.897},"last":1.2729999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.3640000000000001,"15min":1.3859999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.7090000000000001,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.294,"15min":1.3240000000000001},"min":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.544,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.0580000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.248,"15min":1.2669999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.48,"5min":2.161,"15min":2.161},"last":1.23},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2909999999999999,"15min":1.2849999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.663,"5min":1.704,"15min":1.704},"last":0.94099999999999995}]},{"osd":4,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.165,"15min":1.153},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.329,"15min":1.3360000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6140000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.421}]},{"osd":5,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.373,"15min":1.367},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.347,"15min":1.3460000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.452}]},{"osd":6,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3680000000000001,"15min":1.367},"min":{"1min":1.111,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.6359999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.4159999999999999,"15min":1.4259999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.651,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.3180000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":700416,"data_stored":312833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.1040000000000001,"15min":1.129},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.369,"5min":1.49,"15min":1.49},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.264,"15min":1.3140000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5660000000000001,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":1.351}]},{"osd":1,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.2869999999999999,"15min":1.3300000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.304,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.194,"15min":1.212},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.52,"5min":1.623,"15min":1.623},"last":1.018}]},{"osd":2,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.2969999999999999,"15min":1.329},"min":{"1min":0.82099999999999995,"5min":0.745,"15min":0.745},"max":{"1min":1.6519999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.179},{"interface":"front","average":{"1min":1.119,"5min":1.2729999999999999,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.4379999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.395}]},{"osd":3,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3879999999999999,"15min":1.4099999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.8280000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.4319999999999999,"15min":1.462},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.506,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.476}]},{"osd":4,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2709999999999999,"15min":1.282},"min":{"1min":0.746,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5780000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.298,"15min":1.3169999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6100000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.91300000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5249999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.419,"15min":1.4430000000000001},"min":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6679999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.306}]},{"osd":7,"last update":"Fri Sep 13 16:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.514,"15min":1.5369999999999999},"min":{"1min":1.115,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6120000000000001,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.516},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4870000000000001,"15min":1.5209999999999999},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.587,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.8280000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":692224,"data_stored":312181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1779999999999999,"15min":1.1799999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.581,"5min":2.222,"15min":2.222},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.105,"15min":1.095},"min":{"1min":0.78100000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.1160000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.4339999999999999,"15min":1.4530000000000001},"min":{"1min":0.85499999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.6779999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.288},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3460000000000001,"15min":1.349},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.72,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.2589999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.26,"15min":1.2689999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8140000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.266},"min":{"1min":0.90000000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.4770000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.3129999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.407,"15min":1.4399999999999999},"min":{"1min":0.90200000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.823,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3280000000000001,"15min":1.3400000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.7250000000000001,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.0840000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.212,"15min":1.214},"min":{"1min":0.81899999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.655,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.343,"15min":1.3460000000000001},"min":{"1min":0.96199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.649,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.415}]},{"osd":6,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.3100000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.873,"15min":0.873},"max":{"1min":1.552,"5min":1.887,"15min":1.887},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.282,"15min":1.274},"min":{"1min":0.94999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5800000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.179}]},{"osd":7,"last update":"Fri Sep 13 16:52:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.5,"15min":1.514},"min":{"1min":1.109,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7549999999999999,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.466,"15min":1.48},"min":{"1min":1.052,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.704,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.335}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":696320,"data_stored":312763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0820000000000001,"15min":1.1000000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.266,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1,"5min":1.2190000000000001,"15min":1.2549999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3169999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.1659999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.28,"15min":1.3009999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.474,"5min":1.919,"15min":1.919},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.1950000000000001,"15min":1.206},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.52,"5min":1.99,"15min":1.99},"last":1.236}]},{"osd":2,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3009999999999999,"15min":1.327},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4710000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2130000000000001,"15min":1.2350000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.391,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.2030000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.4219999999999999,"15min":1.454},"min":{"1min":1.0289999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5609999999999999,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.405},{"interface":"front","average":{"1min":1.248,"5min":1.4570000000000001,"15min":1.492},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.5549999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.3819999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.3120000000000001,"15min":1.3520000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3129999999999999,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.3720000000000001,"15min":1.3979999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.546,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.488}]},{"osd":6,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.4910000000000001,"15min":1.5289999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5940000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.129},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.5760000000000001,"15min":1.6259999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.546,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.3340000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.5269999999999999,"15min":1.5609999999999999},"min":{"1min":1.135,"5min":1.107,"15min":1.107},"max":{"1min":1.6040000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.452},{"interface":"front","average":{"1min":1.339,"5min":1.6200000000000001,"15min":1.667},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.6140000000000001,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.4319999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":313767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.995,"15min":1.0169999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4990000000000001,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.97899999999999998,"15min":0.98899999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.1060000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.032,"15min":1.0549999999999999},"min":{"1min":0.66400000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.391,"5min":1.931,"15min":1.931},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.052,"5min":1.1759999999999999,"15min":1.196},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.464,"5min":1.849,"15min":1.849},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.3069999999999999,"15min":1.339},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.702,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.995},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.369,"15min":1.403},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.657,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.76200000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.4259999999999999,"15min":1.4630000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.599,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.474},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.492,"15min":1.518},"min":{"1min":1.0980000000000001,"5min":0.88,"15min":0.88},"max":{"1min":1.972,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.298}]},{"osd":5,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.609,"15min":1.645},"min":{"1min":0.90800000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.0569999999999999,"5min":2.79,"15min":2.79},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.5900000000000001,"15min":1.6359999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.71,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.655,"15min":1.7},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.8520000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.284},{"interface":"front","average":{"1min":1.298,"5min":1.579,"15min":1.6259999999999999},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.649,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5740000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.649,"15min":1.6839999999999999},"min":{"1min":1.103,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.012,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.71,"15min":1.7509999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":2.1320000000000001,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.5489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":313745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.93799999999999994,"15min":0.95299999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.113,"15min":1.1299999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.427,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.143}]},{"osd":1,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.94899999999999995,"15min":0.97099999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.351,"5min":1.494,"15min":1.494},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0660000000000001,"15min":1.083},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4079999999999999,"5min":2.028,"15min":2.028},"last":0.90000000000000002}]},{"osd":3,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.198,"15min":1.1879999999999999},"min":{"1min":0.80200000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6870000000000001,"5min":1.948,"15min":1.948},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.161,"15min":1.145},"min":{"1min":0.71099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.0700000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3220000000000001,"15min":1.3260000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.274},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.452,"15min":1.47},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.3049999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.5609999999999999,"15min":1.5880000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.6629999999999998,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.407,"15min":1.423},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3029999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.601}]},{"osd":6,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.548,"15min":1.5740000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.671},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.5820000000000001,"15min":1.6100000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.2629999999999999,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.3839999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:52:27 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.6439999999999999,"15min":1.6799999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.73,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.5},{"interface":"front","average":{"1min":1.472,"5min":1.6419999999999999,"15min":1.6699999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.496,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.722}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":312341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89200000000000002,"15min":0.91000000000000003},"min":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0389999999999999,"5min":1.804,"15min":1.804},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.99099999999999999,"15min":0.998},"min":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5700000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.89500000000000002}]},{"osd":2,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.212,"5min":1.954,"15min":1.954},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0569999999999999,"15min":1.075},"min":{"1min":0.67200000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3300000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.0389999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.169,"15min":1.167},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7729999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1519999999999999,"15min":1.1559999999999999},"min":{"1min":0.68600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.6950000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.296,"15min":1.3180000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.6439999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.369,"15min":1.381},"min":{"1min":0.94799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.6020000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.5209999999999999,"15min":1.5349999999999999},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.3570000000000002,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.48},{"interface":"front","average":{"1min":1.357,"5min":1.5089999999999999,"15min":1.534},"min":{"1min":1.04,"5min":1,"15min":1},"max":{"1min":1.7130000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.843}]},{"osd":6,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5309999999999999,"15min":1.5449999999999999},"min":{"1min":1.1040000000000001,"5min":1.042,"15min":1.042},"max":{"1min":2.508,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.476,"15min":1.502},"min":{"1min":0.95699999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.712,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.7310000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.536,"15min":1.556},"min":{"1min":1.0389999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":1.7450000000000001,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.325},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.5920000000000001,"15min":1.609},"min":{"1min":1.206,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.6030000000000002,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.532}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":902605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.93300000000000005,"15min":0.95599999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1120000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3899999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.83499999999999996}]},{"osd":2,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.044,"15min":1.0469999999999999},"min":{"1min":0.65500000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.47,"5min":1.482,"15min":1.482},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.609,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.024,"15min":1.032},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5489999999999999,"5min":1.653,"15min":1.653},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.149,"5min":1.226,"15min":1.2390000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5920000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.181}]},{"osd":4,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.361,"15min":1.3959999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.5940000000000001,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.26},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3520000000000001,"15min":1.363},"min":{"1min":0.88800000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5309999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.3460000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4750000000000001,"15min":1.5069999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6799999999999999,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.452,"15min":1.4810000000000001},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.7969999999999999,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.371}]},{"osd":6,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5289999999999999,"15min":1.556},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.734,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.5600000000000001,"15min":1.5940000000000001},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.633,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":1.3120000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.6000000000000001,"15min":1.6339999999999999},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":1.698,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.524,"15min":1.548},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":1.7789999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.4379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:53:05.455 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T16:53:05.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T16:53:05.570 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-13T16:53:05.563+0000 7f20f7c00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:53:05.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.679+0000 7f3e0f000700 1 Processor -- start 2024-09-13T16:53:05.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.679+0000 7f3e0f000700 1 -- start start 2024-09-13T16:53:05.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.679+0000 7f3e0f000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3e0806f970 0x7f3e0806fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.679+0000 7f3e0f000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e080cae70 0x7f3e080cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.679+0000 7f3e0f000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e080ce620 con 0x7f3e0815ccc0 2024-09-13T16:53:05.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.679+0000 7f3e0f000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e08070320 con 0x7f3e0806f970 2024-09-13T16:53:05.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.679+0000 7f3e0f000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080ce900 con 0x7f3e080cae70 2024-09-13T16:53:05.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3e0815ccc0 0x7f3e080cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48742/0 (socket says 172.21.15.3:48742) 2024-09-13T16:53:05.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 -- 172.21.15.3:0/1507279561 learned_addr learned my addr 172.21.15.3:0/1507279561 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:05.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e0d000700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e080cae70 0x7f3e080cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07400700 1 --2- 172.21.15.3:0/1507279561 >> v2:172.21.15.157:3300/0 conn(0x7f3e0806f970 0x7f3e0806fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128976813 0 0) 0x7f3e080ce620 con 0x7f3e0815ccc0 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dec0035d0 con 0x7f3e0815ccc0 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1640772551 0 0) 0x7f3dec0035d0 con 0x7f3e0815ccc0 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e080cae70 msgr2=0x7f3e080cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e080cae70 0x7f3e080cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 >> v2:172.21.15.157:3300/0 conn(0x7f3e0806f970 msgr2=0x7f3e0806fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 --2- 172.21.15.3:0/1507279561 >> v2:172.21.15.157:3300/0 conn(0x7f3e0806f970 0x7f3e0806fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e080cebe0 con 0x7f3e0815ccc0 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f3df8001690 con 0x7f3e0815ccc0 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 >> v1:172.21.15.3:6789/0 conn(0x7f3e0815ccc0 legacy=0x7f3e080cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dec003a20 0x7f3dec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dec004870 0x7f3dec004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e080cae70 0x7f3dec005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08070320 con 0x7f3dec004870 2024-09-13T16:53:05.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080ce900 con 0x7f3dec003a20 2024-09-13T16:53:05.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec00ba30 con 0x7f3e080cae70 2024-09-13T16:53:05.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dec003a20 0x7f3dec003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07400700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dec004870 0x7f3dec004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e0d000700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e080cae70 0x7f3dec005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 -- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e080cae70 msgr2=0x7f3dec005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e080cae70 0x7f3dec005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 -- 172.21.15.3:0/1507279561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dec004870 msgr2=0x7f3dec004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dec004870 0x7f3dec004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 -- 172.21.15.3:0/1507279561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec0035d0 con 0x7f3dec003a20 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07400700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dec004870 0x7f3dec004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.683+0000 7f3e07e00700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dec003a20 0x7f3dec003e40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c270 tx=0x7f3dec0058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a2c375c313f0fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8007c80 con 0x7f3dec003a20 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df800ce10 con 0x7f3dec003a20 2024-09-13T16:53:05.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e06a00700 1 -- 172.21.15.3:0/1507279561 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8005bc0 con 0x7f3dec003a20 2024-09-13T16:53:05.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e0f000700 1 -- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dec003a20 msgr2=0x7f3dec003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e0f000700 1 --2- 172.21.15.3:0/1507279561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dec003a20 0x7f3dec003e40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c270 tx=0x7f3dec0058a0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e0d000700 1 -- 172.21.15.3:0/1507279561 reap_dead start 2024-09-13T16:53:05.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e0f000700 1 -- 172.21.15.3:0/1507279561 shutdown_connections 2024-09-13T16:53:05.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e0f000700 1 -- 172.21.15.3:0/1507279561 >> 172.21.15.3:0/1507279561 conn(0x7f3e080b2b00 msgr2=0x7f3e080b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:05.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e0f000700 1 -- 172.21.15.3:0/1507279561 shutdown_connections 2024-09-13T16:53:05.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.687+0000 7f3e0f000700 1 -- 172.21.15.3:0/1507279561 wait complete. 2024-09-13T16:53:05.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0f000700 1 Processor -- start 2024-09-13T16:53:05.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0f000700 1 -- start start 2024-09-13T16:53:05.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0f000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e08143cc0 0x7f3e08140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e081413d0 0x7f3e081440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0f000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e081445e0 0x7f3e081f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0f000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08144a90 con 0x7f3e081445e0 2024-09-13T16:53:05.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081570f0 con 0x7f3e081413d0 2024-09-13T16:53:05.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0f000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08154980 con 0x7f3e08143cc0 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e081445e0 0x7f3e081f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e081445e0 0x7f3e081f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47526/0 (socket says 172.21.15.3:47526) 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 -- 172.21.15.3:0/3327239500 learned_addr learned my addr 172.21.15.3:0/3327239500 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e07e00700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e08143cc0 0x7f3e08140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e07400700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e081413d0 0x7f3e081440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 -- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e08143cc0 msgr2=0x7f3e08140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e08143cc0 0x7f3e08140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 -- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e081413d0 msgr2=0x7f3e081440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e081413d0 0x7f3e081440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 -- 172.21.15.3:0/3327239500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08157580 con 0x7f3e081445e0 2024-09-13T16:53:05.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.691+0000 7f3e0d000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e081445e0 0x7f3e081f16d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3e081608d0 tx=0x7f3e00011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.695+0000 7f3dffe00700 1 -- 172.21.15.3:0/3327239500 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000e040 con 0x7f3e081445e0 2024-09-13T16:53:05.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.695+0000 7f3dffe00700 1 -- 172.21.15.3:0/3327239500 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0000f040 con 0x7f3e081445e0 2024-09-13T16:53:05.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.695+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e081546a0 con 0x7f3e081445e0 2024-09-13T16:53:05.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.695+0000 7f3dffe00700 1 -- 172.21.15.3:0/3327239500 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000f940 con 0x7f3e081445e0 2024-09-13T16:53:05.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.695+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e08156d80 con 0x7f3e081445e0 2024-09-13T16:53:05.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.695+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e081417b0 con 0x7f3e081445e0 2024-09-13T16:53:05.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.699+0000 7f3dffe00700 1 -- 172.21.15.3:0/3327239500 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e0000faa0 con 0x7f3e081445e0 2024-09-13T16:53:05.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.699+0000 7f3dffe00700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3de8066760 0x7f3de8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.699+0000 7f3dffe00700 1 -- 172.21.15.3:0/3327239500 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3e00084690 con 0x7f3e081445e0 2024-09-13T16:53:05.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.699+0000 7f3e07e00700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3de8066760 0x7f3de8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.699+0000 7f3e07e00700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3de8066760 0x7f3de8068c20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3e080b3ad0 tx=0x7f3df8006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.707+0000 7f3dffe00700 1 -- 172.21.15.3:0/3327239500 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e081417b0 con 0x7f3e081445e0 2024-09-13T16:53:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.951+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3e081417b0 con 0x7f3e081445e0 2024-09-13T16:53:05.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.959+0000 7f3dffe00700 1 -- 172.21.15.3:0/3327239500 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3e081417b0 con 0x7f3e081445e0 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3de8066760 msgr2=0x7f3de8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3de8066760 0x7f3de8068c20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3e080b3ad0 tx=0x7f3df8006040 comp rx=0 tx=0).stop 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e081445e0 msgr2=0x7f3e081f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e081445e0 0x7f3e081f16d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3e081608d0 tx=0x7f3e00011f60 comp rx=0 tx=0).stop 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 shutdown_connections 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3de8066760 0x7f3de8068c20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e081445e0 0x7f3e081f16d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e081413d0 0x7f3e081440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 --2- 172.21.15.3:0/3327239500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e08143cc0 0x7f3e08140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 >> 172.21.15.3:0/3327239500 conn(0x7f3e080b2b00 msgr2=0x7f3e080b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:05.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 shutdown_connections 2024-09-13T16:53:05.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:05.963+0000 7f3e0f000700 1 -- 172.21.15.3:0/3327239500 wait complete. 2024-09-13T16:53:05.973 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-13T16:53:05.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.155+0000 7fdc3f200700 1 Processor -- start 2024-09-13T16:53:06.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.155+0000 7fdc3f200700 1 -- start start 2024-09-13T16:53:06.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.155+0000 7fdc3f200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc401443b0 0x7fdc400b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.155+0000 7fdc3f200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc400ce8e0 con 0x7fdc4006ccf0 2024-09-13T16:53:06.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.155+0000 7fdc3f200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc400ccd30 con 0x7fdc40144790 2024-09-13T16:53:06.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.155+0000 7fdc3f200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc400cd010 con 0x7fdc401443b0 2024-09-13T16:53:06.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.155+0000 7fdc3e800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdc4006ccf0 0x7fdc4015b220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48752/0 (socket says 172.21.15.3:48752) 2024-09-13T16:53:06.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.155+0000 7fdc3e800700 1 -- 172.21.15.3:0/1457803066 learned_addr learned my addr 172.21.15.3:0/1457803066 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:06.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3de00700 1 --2- 172.21.15.3:0/1457803066 >> v2:172.21.15.157:3300/0 conn(0x7fdc401443b0 0x7fdc400b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3ca00700 1 -- 172.21.15.3:0/1457803066 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215831941 0 0) 0x7fdc400ce8e0 con 0x7fdc4006ccf0 2024-09-13T16:53:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3ca00700 1 -- 172.21.15.3:0/1457803066 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc240035d0 con 0x7fdc4006ccf0 2024-09-13T16:53:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3ca00700 1 -- 172.21.15.3:0/1457803066 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319906118 0 0) 0x7fdc400ccd30 con 0x7fdc40144790 2024-09-13T16:53:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3ca00700 1 -- 172.21.15.3:0/1457803066 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc400ce8e0 con 0x7fdc40144790 2024-09-13T16:53:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3de00700 1 -- 172.21.15.3:0/1457803066 >> v1:172.21.15.157:6789/0 conn(0x7fdc40144790 legacy=0x7fdc4006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3de00700 1 -- 172.21.15.3:0/1457803066 >> v1:172.21.15.3:6789/0 conn(0x7fdc4006ccf0 legacy=0x7fdc4015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3de00700 1 -- 172.21.15.3:0/1457803066 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400cd2f0 con 0x7fdc401443b0 2024-09-13T16:53:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3de00700 1 --2- 172.21.15.3:0/1457803066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc401443b0 0x7fdc400b39f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009850 tx=0x7fdc30009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e52bd641aed250 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:06.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3ca00700 1 -- 172.21.15.3:0/1457803066 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30011830 con 0x7fdc401443b0 2024-09-13T16:53:06.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3ca00700 1 -- 172.21.15.3:0/1457803066 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc30011990 con 0x7fdc401443b0 2024-09-13T16:53:06.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3ca00700 1 -- 172.21.15.3:0/1457803066 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30011d40 con 0x7fdc401443b0 2024-09-13T16:53:06.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3f200700 1 -- 172.21.15.3:0/1457803066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc401443b0 msgr2=0x7fdc400b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3f200700 1 --2- 172.21.15.3:0/1457803066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc401443b0 0x7fdc400b39f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009850 tx=0x7fdc30009b60 comp rx=0 tx=0).stop 2024-09-13T16:53:06.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3f200700 1 -- 172.21.15.3:0/1457803066 shutdown_connections 2024-09-13T16:53:06.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3f200700 1 --2- 172.21.15.3:0/1457803066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc401443b0 0x7fdc400b39f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3f200700 1 -- 172.21.15.3:0/1457803066 >> 172.21.15.3:0/1457803066 conn(0x7fdc400b0cf0 msgr2=0x7fdc400b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:06.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3f200700 1 -- 172.21.15.3:0/1457803066 shutdown_connections 2024-09-13T16:53:06.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.159+0000 7fdc3f200700 1 -- 172.21.15.3:0/1457803066 wait complete. 2024-09-13T16:53:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.163+0000 7fdc3f200700 1 Processor -- start 2024-09-13T16:53:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.163+0000 7fdc3f200700 1 -- start start 2024-09-13T16:53:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4006ccf0 0x7fdc4013ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3f200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc401443b0 0x7fdc4013f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3f200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4013da90 0x7fdc4013f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3f200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4013e000 con 0x7fdc401443b0 2024-09-13T16:53:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4013e2e0 con 0x7fdc4006ccf0 2024-09-13T16:53:06.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3f200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4013e5c0 con 0x7fdc4013da90 2024-09-13T16:53:06.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc401443b0 0x7fdc4013f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc401443b0 0x7fdc4013f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47532/0 (socket says 172.21.15.3:47532) 2024-09-13T16:53:06.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 -- 172.21.15.3:0/1869656496 learned_addr learned my addr 172.21.15.3:0/1869656496 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3de00700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4006ccf0 0x7fdc4013ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3e800700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4013da90 0x7fdc4013f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 -- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4013da90 msgr2=0x7fdc4013f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4013da90 0x7fdc4013f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 -- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4006ccf0 msgr2=0x7fdc4013ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4006ccf0 0x7fdc4013ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 -- 172.21.15.3:0/1869656496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc300094b0 con 0x7fdc401443b0 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3de00700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4006ccf0 0x7fdc4013ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3d400700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc401443b0 0x7fdc4013f2e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00c730 tx=0x7fdc2c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:06.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc36000700 1 -- 172.21.15.3:0/1869656496 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0052d0 con 0x7fdc401443b0 2024-09-13T16:53:06.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc36000700 1 -- 172.21.15.3:0/1869656496 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c005450 con 0x7fdc401443b0 2024-09-13T16:53:06.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc36000700 1 -- 172.21.15.3:0/1869656496 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0038e0 con 0x7fdc401443b0 2024-09-13T16:53:06.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4013e8a0 con 0x7fdc401443b0 2024-09-13T16:53:06.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.167+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4013b290 con 0x7fdc401443b0 2024-09-13T16:53:06.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.171+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc401423b0 con 0x7fdc401443b0 2024-09-13T16:53:06.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.171+0000 7fdc36000700 1 -- 172.21.15.3:0/1869656496 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc2c003a40 con 0x7fdc401443b0 2024-09-13T16:53:06.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.175+0000 7fdc36000700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdc20066760 0x7fdc20068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.175+0000 7fdc36000700 1 -- 172.21.15.3:0/1869656496 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdc2c083750 con 0x7fdc401443b0 2024-09-13T16:53:06.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.175+0000 7fdc3de00700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdc20066760 0x7fdc20068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.175+0000 7fdc3de00700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdc20066760 0x7fdc20068c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc300090b0 tx=0x7fdc30008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.179+0000 7fdc36000700 1 -- 172.21.15.3:0/1869656496 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdc401423b0 con 0x7fdc401443b0 2024-09-13T16:53:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.415+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc401423b0 con 0x7fdc401443b0 2024-09-13T16:53:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.415+0000 7fdc36000700 1 -- 172.21.15.3:0/1869656496 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4731 (secure 0 0 0) 0x7fdc401423b0 con 0x7fdc401443b0 2024-09-13T16:53:06.423 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:06.423 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5327,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617784475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1617784475},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617784475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5812,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4227223541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4227223541},{"type":"v1","addr":"172.21.15.3:6833","nonce":4227223541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2335957931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2335957931},{"type":"v1","addr":"172.21.15.157:6836","nonce":2335957931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:03.611651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5832},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5832":{"gid":5832,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/291442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":291442393},{"type":"v1","addr":"172.21.15.3:6835","nonce":291442393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdc20066760 msgr2=0x7fdc20068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdc20066760 0x7fdc20068c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc300090b0 tx=0x7fdc30008030 comp rx=0 tx=0).stop 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc401443b0 msgr2=0x7fdc4013f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc401443b0 0x7fdc4013f2e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00c730 tx=0x7fdc2c00ca40 comp rx=0 tx=0).stop 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 shutdown_connections 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdc20066760 0x7fdc20068c20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4013da90 0x7fdc4013f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc401443b0 0x7fdc4013f2e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 --2- 172.21.15.3:0/1869656496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4006ccf0 0x7fdc4013ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 >> 172.21.15.3:0/1869656496 conn(0x7fdc400b0cf0 msgr2=0x7fdc400706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 shutdown_connections 2024-09-13T16:53:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:06.419+0000 7fdc3f200700 1 -- 172.21.15.3:0/1869656496 wait complete. 2024-09-13T16:53:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 30 2024-09-13T16:53:06.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:53:00.113793+0000', 'modified': '2024-09-13T16:53:03.611651+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5832}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5832': {'gid': 5832, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/291442393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 291442393}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 291442393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:53:06.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:53:06.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T16:53:07.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.027+0000 7fc353200700 1 Processor -- start 2024-09-13T16:53:07.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.027+0000 7fc353200700 1 -- start start 2024-09-13T16:53:07.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.031+0000 7fc353200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc35406ccf0 0x7fc3541472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.031+0000 7fc353200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc35406fea0 0x7fc3540702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.031+0000 7fc353200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc354165800 con 0x7fc354146ee0 2024-09-13T16:53:07.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.031+0000 7fc353200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc354070a80 con 0x7fc35406fea0 2024-09-13T16:53:07.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.031+0000 7fc353200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3541626e0 con 0x7fc35406ccf0 2024-09-13T16:53:07.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.031+0000 7fc351e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc354146ee0 0x7fc35406c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48760/0 (socket says 172.21.15.3:48760) 2024-09-13T16:53:07.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.031+0000 7fc351e00700 1 -- 172.21.15.3:0/2317790633 learned_addr learned my addr 172.21.15.3:0/2317790633 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:07.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc352800700 1 --2- 172.21.15.3:0/2317790633 >> v2:172.21.15.3:3300/0 conn(0x7fc35406fea0 0x7fc3540702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc351400700 1 --2- 172.21.15.3:0/2317790633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc35406ccf0 0x7fc3541472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc352800700 1 -- 172.21.15.3:0/2317790633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc35406ccf0 msgr2=0x7fc3541472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc352800700 1 --2- 172.21.15.3:0/2317790633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc35406ccf0 0x7fc3541472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc352800700 1 -- 172.21.15.3:0/2317790633 >> v1:172.21.15.3:6789/0 conn(0x7fc354146ee0 legacy=0x7fc35406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc352800700 1 -- 172.21.15.3:0/2317790633 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354070830 con 0x7fc35406fea0 2024-09-13T16:53:07.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc350a00700 1 -- 172.21.15.3:0/2317790633 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085084255 0 0) 0x7fc354165800 con 0x7fc354146ee0 2024-09-13T16:53:07.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc352800700 1 --2- 172.21.15.3:0/2317790633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406fea0 0x7fc3540702f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009d90 tx=0x7fc34c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbb5227df32a2d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc350a00700 1 -- 172.21.15.3:0/2317790633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00e070 con 0x7fc35406fea0 2024-09-13T16:53:07.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc350a00700 1 -- 172.21.15.3:0/2317790633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c00bb20 con 0x7fc35406fea0 2024-09-13T16:53:07.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc350a00700 1 -- 172.21.15.3:0/2317790633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c012430 con 0x7fc35406fea0 2024-09-13T16:53:07.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc353200700 1 -- 172.21.15.3:0/2317790633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406fea0 msgr2=0x7fc3540702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc353200700 1 --2- 172.21.15.3:0/2317790633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406fea0 0x7fc3540702f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009d90 tx=0x7fc34c00b330 comp rx=0 tx=0).stop 2024-09-13T16:53:07.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc353200700 1 -- 172.21.15.3:0/2317790633 shutdown_connections 2024-09-13T16:53:07.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc353200700 1 --2- 172.21.15.3:0/2317790633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406fea0 0x7fc3540702f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc353200700 1 --2- 172.21.15.3:0/2317790633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc35406ccf0 0x7fc3541472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc353200700 1 -- 172.21.15.3:0/2317790633 >> 172.21.15.3:0/2317790633 conn(0x7fc3540b0cf0 msgr2=0x7fc3540b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc353200700 1 -- 172.21.15.3:0/2317790633 shutdown_connections 2024-09-13T16:53:07.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.035+0000 7fc353200700 1 -- 172.21.15.3:0/2317790633 wait complete. 2024-09-13T16:53:07.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc353200700 1 Processor -- start 2024-09-13T16:53:07.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc353200700 1 -- start start 2024-09-13T16:53:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc353200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406ccf0 0x7fc354145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc353200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35406fea0 0x7fc354146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc353200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc354146ee0 0x7fc3540a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc353200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540abc90 con 0x7fc35406ccf0 2024-09-13T16:53:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc353200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc354159560 con 0x7fc35406fea0 2024-09-13T16:53:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc353200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc354158920 con 0x7fc354146ee0 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406ccf0 0x7fc354145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406ccf0 0x7fc354145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47542/0 (socket says 172.21.15.3:47542) 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 -- 172.21.15.3:0/3117511503 learned_addr learned my addr 172.21.15.3:0/3117511503 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351400700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35406fea0 0x7fc354146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc352800700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc354146ee0 0x7fc3540a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 -- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc354146ee0 msgr2=0x7fc3540a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc354146ee0 0x7fc3540a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 -- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35406fea0 msgr2=0x7fc354146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35406fea0 0x7fc354146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 -- 172.21.15.3:0/3117511503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c009a70 con 0x7fc35406ccf0 2024-09-13T16:53:07.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.039+0000 7fc351e00700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406ccf0 0x7fc354145ff0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c4e0 tx=0x7fc34400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.043+0000 7fc34a000700 1 -- 172.21.15.3:0/3117511503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344004ca0 con 0x7fc35406ccf0 2024-09-13T16:53:07.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.043+0000 7fc34a000700 1 -- 172.21.15.3:0/3117511503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc344004e20 con 0x7fc35406ccf0 2024-09-13T16:53:07.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.043+0000 7fc353200700 1 -- 172.21.15.3:0/3117511503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc354158bc0 con 0x7fc35406ccf0 2024-09-13T16:53:07.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.043+0000 7fc34a000700 1 -- 172.21.15.3:0/3117511503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344007310 con 0x7fc35406ccf0 2024-09-13T16:53:07.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.043+0000 7fc353200700 1 -- 172.21.15.3:0/3117511503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc354159180 con 0x7fc35406ccf0 2024-09-13T16:53:07.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.043+0000 7fc34a000700 1 -- 172.21.15.3:0/3117511503 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc344007470 con 0x7fc35406ccf0 2024-09-13T16:53:07.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.047+0000 7fc34a000700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc334066730 0x7fc334068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.047+0000 7fc34a000700 1 -- 172.21.15.3:0/3117511503 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc344083ed0 con 0x7fc35406ccf0 2024-09-13T16:53:07.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.047+0000 7fc351400700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc334066730 0x7fc334068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.047+0000 7fc351400700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc334066730 0x7fc334068bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc3540aff50 tx=0x7fc340009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.047+0000 7fc35c05a740 1 Processor -- start 2024-09-13T16:53:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.047+0000 7fc35c05a740 1 -- start start 2024-09-13T16:53:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6bda0 0xec6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6c6c0 0xec34dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc35c05a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0xeb93d80 con 0xec4ad00 2024-09-13T16:53:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc35c05a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0xeb91510 con 0xec6bda0 2024-09-13T16:53:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc35c05a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0xeb8ce00 con 0xec6c6c0 2024-09-13T16:53:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec34890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47548/0 (socket says 172.21.15.3:47548) 2024-09-13T16:53:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 -- 172.21.15.3:0/971305259 learned_addr learned my addr 172.21.15.3:0/971305259 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351e00700 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6bda0 0xec6c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc352800700 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6c6c0 0xec34dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 -- 172.21.15.3:0/971305259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6c6c0 msgr2=0xec34dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6c6c0 0xec34dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 -- 172.21.15.3:0/971305259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6bda0 msgr2=0xec6c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6bda0 0xec6c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 -- 172.21.15.3:0/971305259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe9ff960 con 0xec4ad00 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc351400700 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec34890 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc340004500 tx=0x7fc340014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b30dc527cdf9d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc33e600700 1 -- 172.21.15.3:0/971305259 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34001c040 con 0xec4ad00 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc33e600700 1 -- 172.21.15.3:0/971305259 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc340004530 con 0xec4ad00 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.051+0000 7fc33e600700 1 -- 172.21.15.3:0/971305259 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340003ad0 con 0xec4ad00 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- 172.21.15.3:0/971305259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 msgr2=0xec34890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec34890 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc340004500 tx=0x7fc340014000 comp rx=0 tx=0).stop 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- 172.21.15.3:0/971305259 shutdown_connections 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6c6c0 0xec34dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec34890 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 --2- 172.21.15.3:0/971305259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6bda0 0xec6c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- 172.21.15.3:0/971305259 >> 172.21.15.3:0/971305259 conn(0xe9b44d0 msgr2=0xec4dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- 172.21.15.3:0/971305259 shutdown_connections 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- 172.21.15.3:0/971305259 wait complete. 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- start start 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 Processor -- start 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec67580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec72220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0xeb4a730 con 0xec4ad00 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0xeb505b0 con 0xec6c6c0 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0xea38aa0 con 0xec6bda0 2024-09-13T16:53:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc35c05a740 10 client.0 _ll_register_callbacks cb 0xeb86610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc351e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec6c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc351e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec6c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47552/0 (socket says 172.21.15.3:47552) 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.055+0000 7fc351e00700 1 -- 172.21.15.3:0/807405100 learned_addr learned my addr 172.21.15.3:0/807405100 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc352800700 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec72220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351400700 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec67580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351e00700 1 -- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 msgr2=0xec67580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351e00700 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec67580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351e00700 1 -- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 msgr2=0xec72220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351e00700 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec72220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351e00700 1 -- 172.21.15.3:0/807405100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3400041b0 con 0xec4ad00 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351400700 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec67580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351e00700 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec6c180 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc3440822b0 tx=0x7fc3440077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe2c0a4e023e6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc332a00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc332a00700 1 -- 172.21.15.3:0/807405100 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3440544a0 con 0xec4ad00 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc332a00700 1 -- 172.21.15.3:0/807405100 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc344054db0 con 0xec4ad00 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 1 -- 172.21.15.3:0/807405100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe9c11e0 con 0xec4ad00 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc332a00700 1 -- 172.21.15.3:0/807405100 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc344058be0 con 0xec4ad00 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 10 client.5967 fetch_fsmap learned FSMap version 30 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 1 -- 172.21.15.3:0/807405100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0xeb53220 con 0xec4ad00 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc332a00700 1 -- 172.21.15.3:0/807405100 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe9c11e0 con 0xec4ad00 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc332a00700 1 -- 172.21.15.3:0/807405100 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc34407f020 con 0xec4ad00 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 10 client.5967 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 10 client.5967 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 20 client.5967 populate_metadata read hostname 'smithi003' 2024-09-13T16:53:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] conn(0xec24100 0xec244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 4 client.5967 mds_command: new command op to 5832 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:53:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc35c05a740 1 -- 172.21.15.3:0/807405100 --> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xea0b460 con 0xec24100 2024-09-13T16:53:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351400700 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] conn(0xec24100 0xec244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc351400700 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] conn(0xec24100 0xec244e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223555e6ae606e84 server_cookie=e107348abbfbc87e in_seq=0 out_seq=0 2024-09-13T16:53:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.059+0000 7fc332a00700 10 client.5967 ms_handle_connect on v2:172.21.15.3:6834/291442393 2024-09-13T16:53:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc332a00700 1 -- 172.21.15.3:0/807405100 <== mds.0 v2:172.21.15.3:6834/291442393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0xea0b460 con 0xec24100 2024-09-13T16:53:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc332a00700 10 client.5967 handle_command_reply: tid=0 2024-09-13T16:53:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 client.5967 shutdown 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 -- 172.21.15.3:0/807405100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 msgr2=0xec6c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec6c180 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc3440822b0 tx=0x7fc3440077d0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 -- 172.21.15.3:0/807405100 shutdown_connections 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] conn(0xec24100 0xec244e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec72220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec67580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 --2- 172.21.15.3:0/807405100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec6c180 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc332a00700 0 client.5967 ms_handle_reset on v2:172.21.15.3:6834/291442393 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 -- 172.21.15.3:0/807405100 >> 172.21.15.3:0/807405100 conn(0xe9b44d0 msgr2=0xec47780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 -- 172.21.15.3:0/807405100 shutdown_connections 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 1 -- 172.21.15.3:0/807405100 wait complete. 2024-09-13T16:53:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.063+0000 7fc35c05a740 20 client.5967 trim_cache size 0 max 16384 2024-09-13T16:53:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc35c05a740 1 Processor -- start 2024-09-13T16:53:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc35c05a740 1 -- start start 2024-09-13T16:53:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec6caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec6cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc35c05a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0xebbbec0 con 0xec4ad00 2024-09-13T16:53:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc35c05a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0xeb2b470 con 0xec6c6c0 2024-09-13T16:53:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc35c05a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0xec6c3a0 con 0xec6bda0 2024-09-13T16:53:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec49960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47556/0 (socket says 172.21.15.3:47556) 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351e00700 1 -- 172.21.15.3:0/2611774729 learned_addr learned my addr 172.21.15.3:0/2611774729 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc352800700 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec6cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351400700 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec6caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351e00700 1 -- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 msgr2=0xec6caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351e00700 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec6caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351e00700 1 -- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 msgr2=0xec6cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351e00700 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec6cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351e00700 1 -- 172.21.15.3:0/2611774729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xeb39450 con 0xec4ad00 2024-09-13T16:53:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.087+0000 7fc351400700 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec6caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc351e00700 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec49960 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc3440517d0 tx=0x7fc344069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbd28fcf8a8b8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc33f000700 1 -- 172.21.15.3:0/2611774729 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344058be0 con 0xec4ad00 2024-09-13T16:53:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc33f000700 1 -- 172.21.15.3:0/2611774729 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3440544a0 con 0xec4ad00 2024-09-13T16:53:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc33f000700 1 -- 172.21.15.3:0/2611774729 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34405cf10 con 0xec4ad00 2024-09-13T16:53:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 -- 172.21.15.3:0/2611774729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 msgr2=0xec49960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec49960 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc3440517d0 tx=0x7fc344069000 comp rx=0 tx=0).stop 2024-09-13T16:53:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 -- 172.21.15.3:0/2611774729 shutdown_connections 2024-09-13T16:53:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec6cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec6caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2611774729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xec49960 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 -- 172.21.15.3:0/2611774729 >> 172.21.15.3:0/2611774729 conn(0xe9b44d0 msgr2=0xec48050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 -- 172.21.15.3:0/2611774729 shutdown_connections 2024-09-13T16:53:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 -- 172.21.15.3:0/2611774729 wait complete. 2024-09-13T16:53:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 -- start start 2024-09-13T16:53:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.091+0000 7fc35c05a740 1 Processor -- start 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xeccffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec8b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc35c05a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec8bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc35c05a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0xec67b00 con 0xec4ad00 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc35c05a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0xeb4c9d0 con 0xec6c6c0 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc35c05a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0xec1c320 con 0xec6bda0 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc35c05a740 10 client.0 _ll_register_callbacks cb 0xebe3710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xeccffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xeccffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47568/0 (socket says 172.21.15.3:47568) 2024-09-13T16:53:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 -- 172.21.15.3:0/2781489648 learned_addr learned my addr 172.21.15.3:0/2781489648 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc352800700 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec8bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351400700 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec8b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 -- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 msgr2=0xec8b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec8b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 -- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 msgr2=0xec8bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec8bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 -- 172.21.15.3:0/2781489648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xe9ff960 con 0xec4ad00 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc352800700 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec8bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc351e00700 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xeccffa0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc3440546a0 tx=0x7fc34401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a661da5fab0ab5f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc332a00700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:53:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc332a00700 1 -- 172.21.15.3:0/2781489648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344058be0 con 0xec4ad00 2024-09-13T16:53:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc35c05a740 1 -- 172.21.15.3:0/2781489648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc340004530 con 0xec4ad00 2024-09-13T16:53:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.095+0000 7fc332a00700 1 -- 172.21.15.3:0/2781489648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc344059e30 con 0xec4ad00 2024-09-13T16:53:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc332a00700 1 -- 172.21.15.3:0/2781489648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3440011f0 con 0xec4ad00 2024-09-13T16:53:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 10 client.5973 fetch_fsmap learned FSMap version 30 2024-09-13T16:53:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 -- 172.21.15.3:0/2781489648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0xeb53220 con 0xec4ad00 2024-09-13T16:53:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc332a00700 1 -- 172.21.15.3:0/2781489648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc340004530 con 0xec4ad00 2024-09-13T16:53:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc332a00700 1 -- 172.21.15.3:0/2781489648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc34407f040 con 0xec4ad00 2024-09-13T16:53:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 10 client.5973 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T16:53:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 10 client.5973 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T16:53:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 20 client.5973 populate_metadata read hostname 'smithi003' 2024-09-13T16:53:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] conn(0xec8c200 0xec8c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 4 client.5973 mds_command: new command op to 5832 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:53:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 -- 172.21.15.3:0/2781489648 --> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] -- command(tid 0: {"prefix": "status"}) v1 -- 0xea301e0 con 0xec8c200 2024-09-13T16:53:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc351400700 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] conn(0xec8c200 0xec8c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc351400700 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] conn(0xec8c200 0xec8c5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bcbf9e969dc2c0 server_cookie=13649da917c36365 in_seq=0 out_seq=0 2024-09-13T16:53:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc332a00700 10 client.5973 ms_handle_connect on v2:172.21.15.3:6834/291442393 2024-09-13T16:53:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc332a00700 1 -- 172.21.15.3:0/2781489648 <== mds.0 v2:172.21.15.3:6834/291442393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x7fc340018150 con 0xec8c200 2024-09-13T16:53:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc332a00700 10 client.5973 handle_command_reply: tid=0 2024-09-13T16:53:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 client.5973 shutdown 2024-09-13T16:53:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 -- 172.21.15.3:0/2781489648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 msgr2=0xeccffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xeccffa0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc3440546a0 tx=0x7fc34401f040 comp rx=0 tx=0).stop 2024-09-13T16:53:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 -- 172.21.15.3:0/2781489648 shutdown_connections 2024-09-13T16:53:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.3:6834/291442393,v1:172.21.15.3:6835/291442393] conn(0xec8c200 0xec8c5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0xec6c6c0 0xec8bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0xec6bda0 0xec8b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc332a00700 0 client.5973 ms_handle_reset on v2:172.21.15.3:6834/291442393 2024-09-13T16:53:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 --2- 172.21.15.3:0/2781489648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xec4ad00 0xeccffa0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 -- 172.21.15.3:0/2781489648 >> 172.21.15.3:0/2781489648 conn(0xe9b44d0 msgr2=0xec4a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 -- 172.21.15.3:0/2781489648 shutdown_connections 2024-09-13T16:53:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 1 -- 172.21.15.3:0/2781489648 wait complete. 2024-09-13T16:53:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.099+0000 7fc35c05a740 20 client.5973 trim_cache size 0 max 16384 2024-09-13T16:53:07.112 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:53:07.112 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "61456116-a42d-4065-9bf5-a1daf0be594e", 2024-09-13T16:53:07.113 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-13T16:53:07.113 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5832, 2024-09-13T16:53:07.113 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-13T16:53:07.113 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-13T16:53:07.113 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-13T16:53:07.113 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 6.985782639, 2024-09-13T16:53:07.113 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 30, 2024-09-13T16:53:07.113 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 58, 2024-09-13T16:53:07.114 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 58, 2024-09-13T16:53:07.114 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 12.207925267 2024-09-13T16:53:07.114 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:53:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 -- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc334066730 msgr2=0x7fc334068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc334066730 0x7fc334068bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc3540aff50 tx=0x7fc340009240 comp rx=0 tx=0).stop 2024-09-13T16:53:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 -- 172.21.15.3:0/3117511503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406ccf0 msgr2=0x7fc354145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406ccf0 0x7fc354145ff0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c4e0 tx=0x7fc34400c7f0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 -- 172.21.15.3:0/3117511503 shutdown_connections 2024-09-13T16:53:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc334066730 0x7fc334068bf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc354146ee0 0x7fc3540a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35406fea0 0x7fc354146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 --2- 172.21.15.3:0/3117511503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc35406ccf0 0x7fc354145ff0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 -- 172.21.15.3:0/3117511503 >> 172.21.15.3:0/3117511503 conn(0x7fc3540b0cf0 msgr2=0x7fc354162870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 -- 172.21.15.3:0/3117511503 shutdown_connections 2024-09-13T16:53:07.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.103+0000 7fc353200700 1 -- 172.21.15.3:0/3117511503 wait complete. 2024-09-13T16:53:07.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:53:07.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.299+0000 7f985d800700 1 Processor -- start 2024-09-13T16:53:07.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.299+0000 7f985d800700 1 -- start start 2024-09-13T16:53:07.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f985d800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cbc0 0x7f98580adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f985d800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f985815cfa0 0x7f98580ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f985d800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9858070080 0x7f98580ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f985d800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985806a390 con 0x7f9858070080 2024-09-13T16:53:07.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f985d800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985806a670 con 0x7f985815cfa0 2024-09-13T16:53:07.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f985d800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985806a950 con 0x7f985815cbc0 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cbc0 0x7f98580adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f985ce00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9858070080 0x7f98580ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cbc0 0x7f98580adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:40204/0 (socket says 172.21.15.3:40204) 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 -- 172.21.15.3:0/3168391792 learned_addr learned my addr 172.21.15.3:0/3168391792 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f984f400700 1 --2- 172.21.15.3:0/3168391792 >> v2:172.21.15.157:3300/0 conn(0x7f985815cfa0 0x7f98580ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 -- 172.21.15.3:0/3168391792 >> v2:172.21.15.157:3300/0 conn(0x7f985815cfa0 msgr2=0x7f98580ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 --2- 172.21.15.3:0/3168391792 >> v2:172.21.15.157:3300/0 conn(0x7f985815cfa0 0x7f98580ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 -- 172.21.15.3:0/3168391792 >> v2:172.21.15.3:3300/0 conn(0x7f9858070080 msgr2=0x7f98580ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 --2- 172.21.15.3:0/3168391792 >> v2:172.21.15.3:3300/0 conn(0x7f9858070080 0x7f98580ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 -- 172.21.15.3:0/3168391792 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985806ac30 con 0x7f985815cbc0 2024-09-13T16:53:07.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f985ce00700 1 --2- 172.21.15.3:0/3168391792 >> v2:172.21.15.3:3300/0 conn(0x7f9858070080 0x7f98580ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:07.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857e00700 1 --2- 172.21.15.3:0/3168391792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cbc0 0x7f98580adad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98580b1990 tx=0x7f9848009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56463770deac38c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857400700 1 -- 172.21.15.3:0/3168391792 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848011830 con 0x7f985815cbc0 2024-09-13T16:53:07.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857400700 1 -- 172.21.15.3:0/3168391792 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9848011990 con 0x7f985815cbc0 2024-09-13T16:53:07.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.303+0000 7f9857400700 1 -- 172.21.15.3:0/3168391792 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848011d20 con 0x7f985815cbc0 2024-09-13T16:53:07.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 -- 172.21.15.3:0/3168391792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cbc0 msgr2=0x7f98580adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 --2- 172.21.15.3:0/3168391792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cbc0 0x7f98580adad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98580b1990 tx=0x7f9848009bf0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 -- 172.21.15.3:0/3168391792 shutdown_connections 2024-09-13T16:53:07.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 --2- 172.21.15.3:0/3168391792 >> v2:172.21.15.3:3300/0 conn(0x7f9858070080 0x7f98580ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 --2- 172.21.15.3:0/3168391792 >> v2:172.21.15.157:3300/0 conn(0x7f985815cfa0 0x7f98580ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 --2- 172.21.15.3:0/3168391792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cbc0 0x7f98580adad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 -- 172.21.15.3:0/3168391792 >> 172.21.15.3:0/3168391792 conn(0x7f98580b0cf0 msgr2=0x7f98580b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 -- 172.21.15.3:0/3168391792 shutdown_connections 2024-09-13T16:53:07.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 -- 172.21.15.3:0/3168391792 wait complete. 2024-09-13T16:53:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 Processor -- start 2024-09-13T16:53:07.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.307+0000 7f985d800700 1 -- start start 2024-09-13T16:53:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858070080 0x7f9858153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985d800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f985815cbc0 0x7f985815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985d800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cfa0 0x7f98581f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985d800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858156110 con 0x7f985815cbc0 2024-09-13T16:53:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985d800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858160240 con 0x7f9858070080 2024-09-13T16:53:07.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985d800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98581557e0 con 0x7f985815cfa0 2024-09-13T16:53:07.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f985815cbc0 0x7f985815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f985815cbc0 0x7f985815fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47596/0 (socket says 172.21.15.3:47596) 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 -- 172.21.15.3:0/3842741877 learned_addr learned my addr 172.21.15.3:0/3842741877 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f9857e00700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858070080 0x7f9858153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985ce00700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cfa0 0x7f98581f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 -- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cfa0 msgr2=0x7f98581f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cfa0 0x7f98581f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 -- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858070080 msgr2=0x7f9858153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858070080 0x7f9858153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 -- 172.21.15.3:0/3842741877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9850009a70 con 0x7f985815cbc0 2024-09-13T16:53:07.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f984f400700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f985815cbc0 0x7f985815fd00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f984000e870 tx=0x7f984000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f9854c00700 1 -- 172.21.15.3:0/3842741877 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840013680 con 0x7f985815cbc0 2024-09-13T16:53:07.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9848009570 con 0x7f985815cbc0 2024-09-13T16:53:07.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f9854c00700 1 -- 172.21.15.3:0/3842741877 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98400137e0 con 0x7f985815cbc0 2024-09-13T16:53:07.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f9854c00700 1 -- 172.21.15.3:0/3842741877 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840013b90 con 0x7f985815cbc0 2024-09-13T16:53:07.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.311+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9858155da0 con 0x7f985815cbc0 2024-09-13T16:53:07.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.315+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98580ccb60 con 0x7f985815cbc0 2024-09-13T16:53:07.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.315+0000 7f9854c00700 1 -- 172.21.15.3:0/3842741877 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9840013cf0 con 0x7f985815cbc0 2024-09-13T16:53:07.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.319+0000 7f9854c00700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9834066730 0x7f9834068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.319+0000 7f9854c00700 1 -- 172.21.15.3:0/3842741877 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9840084110 con 0x7f985815cbc0 2024-09-13T16:53:07.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.319+0000 7f9857e00700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9834066730 0x7f9834068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.319+0000 7f9857e00700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9834066730 0x7f9834068bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9848000c00 tx=0x7f9848008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.327+0000 7f9854c00700 1 -- 172.21.15.3:0/3842741877 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f98400512b0 con 0x7f985815cbc0 2024-09-13T16:53:07.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.555+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f985806d4d0 con 0x7f985815cbc0 2024-09-13T16:53:07.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.555+0000 7f9854c00700 1 -- 172.21.15.3:0/3842741877 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7f98400548d0 con 0x7f985815cbc0 2024-09-13T16:53:07.562 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T16:53:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.555+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9834066730 msgr2=0x7f9834068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.555+0000 7f985d800700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9834066730 0x7f9834068bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9848000c00 tx=0x7f9848008030 comp rx=0 tx=0).stop 2024-09-13T16:53:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.555+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f985815cbc0 msgr2=0x7f985815fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.555+0000 7f985d800700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f985815cbc0 0x7f985815fd00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f984000e870 tx=0x7f984000eb80 comp rx=0 tx=0).stop 2024-09-13T16:53:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.559+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 shutdown_connections 2024-09-13T16:53:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.559+0000 7f985d800700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9834066730 0x7f9834068bf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.559+0000 7f985d800700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985815cfa0 0x7f98581f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.559+0000 7f985d800700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f985815cbc0 0x7f985815fd00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.559+0000 7f985d800700 1 --2- 172.21.15.3:0/3842741877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858070080 0x7f9858153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.559+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 >> 172.21.15.3:0/3842741877 conn(0x7f98580b0cf0 msgr2=0x7f9858071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.559+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 shutdown_connections 2024-09-13T16:53:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:07.559+0000 7f985d800700 1 -- 172.21.15.3:0/3842741877 wait complete. 2024-09-13T16:53:07.586 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T16:53:07.586 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:53:07.586 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-13T16:53:07.586 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T16:53:07.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T16:53:07.587 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T16:53:07.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T16:53:07.587 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:53:07.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:53:07.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:07.587 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-13T16:53:07.604 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:53:07.605 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:53:07.605 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:53:07.605 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:53:07.605 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-13T16:53:07.605 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:53:07.605 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:53:07.605 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-13T16:53:07.605 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-13T16:53:07.606 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:4d:e8:d2:67 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether f2:5f:fd:43:95:5c brd ff:ff:ff:ff:ff:ff 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:53:07.607 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::4cad:7aff:fedc:96c0/64 scope link 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether f2:5f:fd:43:95:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f05f:fdff:fe43:955c/64 scope link 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether fa:f0:e1:81:09:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f8f0:e1ff:fe81:969/64 scope link 2024-09-13T16:53:07.608 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:53:07.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:07.609 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:53:07.683 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-13T16:53:07.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:07.686 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-13T16:53:07.749 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-13T16:53:07.749 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:53:07.749 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T16:53:07.749 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:53:07.749 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T16:53:07.749 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:53:07.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:07.750 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-09-13T16:53:07.750 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-13T16:53:07.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T16:53:07.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T16:53:07.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T16:53:07.751 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-13T16:53:07.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:07.885 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:53:07.902 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T16:53:07.902 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T16:53:07.903 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:53:07.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:07.903 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-13T16:53:07.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:53:07.982 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T16:53:08.043 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:53:08.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:53:08.044 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:53:08.105 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-13T16:53:08.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:08.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:53:08.228 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:53:08.229 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:53:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-13T16:53:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-13T16:53:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-13T16:53:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 0@61456116-a42d-4065-9bf5-a1daf0be594e.cephfs=/ 2024-09-13T16:53:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.3:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-13T16:53:08.262 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T16:53:08.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:08.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:53:08.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:53:08.292 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T16:53:08.368 INFO:teuthology.orchestra.run.smithi003.stdout:client5982 2024-09-13T16:53:08.369 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:53:08.369 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:53:08.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:53:08.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-13T16:53:08.559 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-13T16:53:08.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.561 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:53:08.562 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-13T16:53:08.563 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-13T16:53:08.564 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:53:08.565 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:53:08.566 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-13T16:53:08.567 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.568 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:53:08.569 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-13T16:53:08.570 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:53:08.571 INFO:teuthology.orchestra.run.smithi003.stdout: osd_deep_scrub_large_omap_object_key_threshold = 1 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-09-13T16:53:08.572 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-09-13T16:53:08.573 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:53:08.574 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.575 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-09-13T16:53:08.576 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-09-13T16:53:08.577 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-09-13T16:53:08.577 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.577 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-09-13T16:53:08.577 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-09-13T16:53:08.577 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.577 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-09-13T16:53:08.578 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:53:08.579 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-09-13T16:53:08.580 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.581 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on pool no app = False 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:53:08.582 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-09-13T16:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:53:08.584 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-09-13T16:53:08.584 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-09-13T16:53:08.584 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-09-13T16:53:08.584 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-09-13T16:53:08.584 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:53:08.584 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:53:08.584 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:53:08.584 INFO:teuthology.orchestra.run.smithi157.stdout: osd_deep_scrub_large_omap_object_key_threshold = 1 2024-09-13T16:53:08.586 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:53:08.586 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T16:53:08.586 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:53:08.586 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:53:08.586 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T16:53:08.586 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T16:53:08.587 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T16:53:08.587 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:53:08.587 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:53:08.587 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T16:53:08.587 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:53:08.588 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:53:08.588 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:53:08.588 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T16:53:08.588 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:53:08.588 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:53:08.609 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:53:08.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:53:08.611 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T16:53:08.612 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T16:53:08.614 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:53:08.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:53:08.616 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:53:08.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:53:08.661 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:53:08.661 DEBUG:teuthology.parallel:result is None 2024-09-13T16:53:08.662 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:53:08.663 DEBUG:teuthology.parallel:result is None 2024-09-13T16:53:08.664 INFO:tasks.ceph.mds.c:Started 2024-09-13T16:53:08.664 DEBUG:teuthology.parallel:result is None 2024-09-13T16:53:08.665 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:53:08.665 DEBUG:teuthology.parallel:result is None 2024-09-13T16:53:08.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:08.774 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-13T16:53:08.775 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-13T16:53:08.777 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-13T16:53:08.784 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-13T16:53:08.779+0000 7f9c4ac00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:53:08.792 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-13T16:53:08.805 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-13T16:53:08.799+0000 7f9c47a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:53:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41e00700 1 Processor -- start 2024-09-13T16:53:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41e00700 1 -- start start 2024-09-13T16:53:08.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a3c0cefc0 0x7f3a3c165bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:08.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c06dd60 con 0x7f3a3c0cc060 2024-09-13T16:53:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c06f1a0 con 0x7f3a3c15aca0 2024-09-13T16:53:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3c0b3250 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a3c0cefc0 0x7f3a3c165bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a3c0cefc0 0x7f3a3c165bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47644/0 (socket says 172.21.15.3:47644) 2024-09-13T16:53:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41400700 1 -- 172.21.15.3:0/820224291 learned_addr learned my addr 172.21.15.3:0/820224291 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41400700 1 -- 172.21.15.3:0/820224291 >> v1:172.21.15.157:6789/0 conn(0x7f3a3c15aca0 legacy=0x7f3a3c06ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41400700 1 -- 172.21.15.3:0/820224291 >> v1:172.21.15.3:6789/0 conn(0x7f3a3c0cc060 legacy=0x7f3a3c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41400700 1 -- 172.21.15.3:0/820224291 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0cc7b0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a41400700 1 --2- 172.21.15.3:0/820224291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c165bc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009e50 tx=0x7f3a3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8460ffd7e721193 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.855+0000 7f3a3b400700 1 -- 172.21.15.3:0/820224291 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400e070 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a3b400700 1 -- 172.21.15.3:0/820224291 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3400b9c0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a3b400700 1 -- 172.21.15.3:0/820224291 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400bd70 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 -- 172.21.15.3:0/820224291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 msgr2=0x7f3a3c165bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 --2- 172.21.15.3:0/820224291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c165bc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009e50 tx=0x7f3a3400b330 comp rx=0 tx=0).stop 2024-09-13T16:53:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 -- 172.21.15.3:0/820224291 shutdown_connections 2024-09-13T16:53:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 --2- 172.21.15.3:0/820224291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c165bc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 -- 172.21.15.3:0/820224291 >> 172.21.15.3:0/820224291 conn(0x7f3a3c0b24c0 msgr2=0x7f3a3c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 -- 172.21.15.3:0/820224291 shutdown_connections 2024-09-13T16:53:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 -- 172.21.15.3:0/820224291 wait complete. 2024-09-13T16:53:08.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 Processor -- start 2024-09-13T16:53:08.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.859+0000 7f3a41e00700 1 -- start start 2024-09-13T16:53:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0cc060 0x7f3a3c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a3c15aca0 0x7f3a3c06b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c15a4b0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c0c75f0 con 0x7f3a3c0cc060 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c1f3ca0 con 0x7f3a3c15aca0 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a40a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0cc060 0x7f3a3c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c1f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47654/0 (socket says 172.21.15.3:47654) 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 -- 172.21.15.3:0/4021806944 learned_addr learned my addr 172.21.15.3:0/4021806944 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41400700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a3c15aca0 0x7f3a3c06b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 -- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a3c15aca0 msgr2=0x7f3a3c06b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a3c15aca0 0x7f3a3c06b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 -- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0cc060 msgr2=0x7f3a3c159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0cc060 0x7f3a3c159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 -- 172.21.15.3:0/4021806944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a34009b30 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a3be00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c1f3760 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00c500 tx=0x7f3a2c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a38c00700 1 -- 172.21.15.3:0/4021806944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c0052d0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a38c00700 1 -- 172.21.15.3:0/4021806944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c005430 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a38c00700 1 -- 172.21.15.3:0/4021806944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c0055a0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c15e0b0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.863+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c0c46c0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.867+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c15b080 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.867+0000 7f3a38c00700 1 -- 172.21.15.3:0/4021806944 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a2c0038e0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.871+0000 7f3a38c00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a20066760 0x7f3a20068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:08.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.871+0000 7f3a38c00700 1 -- 172.21.15.3:0/4021806944 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f3a2c0836a0 con 0x7f3a3c0cefc0 2024-09-13T16:53:08.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.871+0000 7f3a40a00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a20066760 0x7f3a20068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.871+0000 7f3a40a00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a20066760 0x7f3a20068c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3a30000f60 tx=0x7f3a30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:08.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:08.875+0000 7f3a38c00700 1 -- 172.21.15.3:0/4021806944 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3a2c050810 con 0x7f3a3c0cefc0 2024-09-13T16:53:09.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.107+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a3c070500 con 0x7f3a3c0cefc0 2024-09-13T16:53:09.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.107+0000 7f3a38c00700 1 -- 172.21.15.3:0/4021806944 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f3a2c053e30 con 0x7f3a3c0cefc0 2024-09-13T16:53:09.115 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:09.115 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:08.815155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a20066760 msgr2=0x7f3a20068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a20066760 0x7f3a20068c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3a30000f60 tx=0x7f3a30009040 comp rx=0 tx=0).stop 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 msgr2=0x7f3a3c1f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c1f3760 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00c500 tx=0x7f3a2c00c810 comp rx=0 tx=0).stop 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 shutdown_connections 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3a20066760 0x7f3a20068c20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a3c15aca0 0x7f3a3c06b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3c0cefc0 0x7f3a3c1f3760 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 --2- 172.21.15.3:0/4021806944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0cc060 0x7f3a3c159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 >> 172.21.15.3:0/4021806944 conn(0x7f3a3c0b24c0 msgr2=0x7f3a3c0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:09.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 shutdown_connections 2024-09-13T16:53:09.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:09.111+0000 7f3a41e00700 1 -- 172.21.15.3:0/4021806944 wait complete. 2024-09-13T16:53:09.126 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 34 2024-09-13T16:53:09.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:53:00.113793+0000', 'modified': '2024-09-13T16:53:08.815155+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5983}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5983': {'gid': 5983, 'name': 'c', 'rank': 0, 'incarnation': 34, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.3:6835/2114372521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 2114372521}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 2114372521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:53:09.543 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5983:up:replay 2024-09-13T16:53:10.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.723+0000 7f3982a00700 1 Processor -- start 2024-09-13T16:53:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.723+0000 7f3982a00700 1 -- start start 2024-09-13T16:53:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3982a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c15ed00 0x7f397c1621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3982a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f397c0c3980 0x7f397c0c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3982a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0c4320 con 0x7f397c164d30 2024-09-13T16:53:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3982a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397c0b01d0 con 0x7f397c0c3980 2024-09-13T16:53:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3982a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c162700 con 0x7f397c15ed00 2024-09-13T16:53:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3980c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f397c0c3980 0x7f397c0c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3981600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c15ed00 0x7f397c1621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3980c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f397c0c3980 0x7f397c0c3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:52118/0 (socket says 172.21.15.3:52118) 2024-09-13T16:53:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3980c00700 1 -- 172.21.15.3:0/3261045959 learned_addr learned my addr 172.21.15.3:0/3261045959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3982000700 1 --1- 172.21.15.3:0/3261045959 >> v1:172.21.15.157:6789/0 conn(0x7f397c164d30 0x7f397c1651e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:54138/0 (socket says 172.21.15.3:54138) 2024-09-13T16:53:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3981600700 1 -- 172.21.15.3:0/3261045959 >> v2:172.21.15.157:3300/0 conn(0x7f397c0c3980 msgr2=0x7f397c0c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3981600700 1 --2- 172.21.15.3:0/3261045959 >> v2:172.21.15.157:3300/0 conn(0x7f397c0c3980 0x7f397c0c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3981600700 1 -- 172.21.15.3:0/3261045959 >> v1:172.21.15.157:6789/0 conn(0x7f397c164d30 legacy=0x7f397c1651e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3981600700 1 -- 172.21.15.3:0/3261045959 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0afef0 con 0x7f397c15ed00 2024-09-13T16:53:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f3981600700 1 --2- 172.21.15.3:0/3261045959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c15ed00 0x7f397c1621c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3970009850 tx=0x7f3970009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a2ed4e7287d74ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f397be00700 1 -- 172.21.15.3:0/3261045959 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970011830 con 0x7f397c15ed00 2024-09-13T16:53:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f397be00700 1 -- 172.21.15.3:0/3261045959 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3970011990 con 0x7f397c15ed00 2024-09-13T16:53:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.727+0000 7f397be00700 1 -- 172.21.15.3:0/3261045959 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970011d40 con 0x7f397c15ed00 2024-09-13T16:53:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.731+0000 7f3982a00700 1 -- 172.21.15.3:0/3261045959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c15ed00 msgr2=0x7f397c1621c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.731+0000 7f3982a00700 1 --2- 172.21.15.3:0/3261045959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c15ed00 0x7f397c1621c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3970009850 tx=0x7f3970009b60 comp rx=0 tx=0).stop 2024-09-13T16:53:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.731+0000 7f3982a00700 1 -- 172.21.15.3:0/3261045959 shutdown_connections 2024-09-13T16:53:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.731+0000 7f3982a00700 1 --2- 172.21.15.3:0/3261045959 >> v2:172.21.15.157:3300/0 conn(0x7f397c0c3980 0x7f397c0c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.731+0000 7f3982a00700 1 --2- 172.21.15.3:0/3261045959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c15ed00 0x7f397c1621c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.731+0000 7f3982a00700 1 -- 172.21.15.3:0/3261045959 >> 172.21.15.3:0/3261045959 conn(0x7f397c0b24c0 msgr2=0x7f397c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.731+0000 7f3982a00700 1 -- 172.21.15.3:0/3261045959 shutdown_connections 2024-09-13T16:53:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.731+0000 7f3982a00700 1 -- 172.21.15.3:0/3261045959 wait complete. 2024-09-13T16:53:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982a00700 1 Processor -- start 2024-09-13T16:53:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982a00700 1 -- start start 2024-09-13T16:53:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c0c3980 0x7f397c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f397c15ed00 0x7f397c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f397c164d30 0x7f397c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0b4cb0 con 0x7f397c164d30 2024-09-13T16:53:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0b5820 con 0x7f397c15ed00 2024-09-13T16:53:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c15cee0 con 0x7f397c0c3980 2024-09-13T16:53:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f397c164d30 0x7f397c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3981600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c0c3980 0x7f397c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f397c164d30 0x7f397c0b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47664/0 (socket says 172.21.15.3:47664) 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3981600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c0c3980 0x7f397c15d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:40270/0 (socket says 172.21.15.3:40270) 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982000700 1 -- 172.21.15.3:0/1069798053 learned_addr learned my addr 172.21.15.3:0/1069798053 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3980c00700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f397c15ed00 0x7f397c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982000700 1 -- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c0c3980 msgr2=0x7f397c15d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982000700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c0c3980 0x7f397c15d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982000700 1 -- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f397c15ed00 msgr2=0x7f397c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982000700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f397c15ed00 0x7f397c15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.735+0000 7f3982000700 1 -- 172.21.15.3:0/1069798053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700094b0 con 0x7f397c164d30 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.739+0000 7f3982000700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f397c164d30 0x7f397c0b41c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f397400cc00 tx=0x7f397400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.739+0000 7f3979600700 1 -- 172.21.15.3:0/1069798053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400aa40 con 0x7f397c164d30 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.739+0000 7f3979600700 1 -- 172.21.15.3:0/1069798053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397400aba0 con 0x7f397c164d30 2024-09-13T16:53:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.739+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c0b4700 con 0x7f397c164d30 2024-09-13T16:53:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.739+0000 7f3979600700 1 -- 172.21.15.3:0/1069798053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974007070 con 0x7f397c164d30 2024-09-13T16:53:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.739+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c0b51d0 con 0x7f397c164d30 2024-09-13T16:53:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.739+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c15d8d0 con 0x7f397c164d30 2024-09-13T16:53:10.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.743+0000 7f3979600700 1 -- 172.21.15.3:0/1069798053 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f39740071d0 con 0x7f397c164d30 2024-09-13T16:53:10.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.743+0000 7f3979600700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f395c066730 0x7f395c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:10.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.743+0000 7f3979600700 1 -- 172.21.15.3:0/1069798053 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f3974083eb0 con 0x7f397c164d30 2024-09-13T16:53:10.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.747+0000 7f3981600700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f395c066730 0x7f395c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:10.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.747+0000 7f3981600700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f395c066730 0x7f395c068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f39700090b0 tx=0x7f3970008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.751+0000 7f3979600700 1 -- 172.21.15.3:0/1069798053 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3974051170 con 0x7f397c164d30 2024-09-13T16:53:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.983+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f397c06d2b0 con 0x7f397c164d30 2024-09-13T16:53:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.983+0000 7f3979600700 1 -- 172.21.15.3:0/1069798053 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f3974017070 con 0x7f397c164d30 2024-09-13T16:53:10.990 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:10.990 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:10.833834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f395c066730 msgr2=0x7f395c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f395c066730 0x7f395c068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f39700090b0 tx=0x7f3970008030 comp rx=0 tx=0).stop 2024-09-13T16:53:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f397c164d30 msgr2=0x7f397c0b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f397c164d30 0x7f397c0b41c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f397400cc00 tx=0x7f397400cf10 comp rx=0 tx=0).stop 2024-09-13T16:53:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 shutdown_connections 2024-09-13T16:53:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f395c066730 0x7f395c068bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f397c164d30 0x7f397c0b41c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f397c15ed00 0x7f397c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 --2- 172.21.15.3:0/1069798053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f397c0c3980 0x7f397c15d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 >> 172.21.15.3:0/1069798053 conn(0x7f397c0b24c0 msgr2=0x7f397c0b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 shutdown_connections 2024-09-13T16:53:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:10.987+0000 7f3982a00700 1 -- 172.21.15.3:0/1069798053 wait complete. 2024-09-13T16:53:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 36 2024-09-13T16:53:11.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:53:00.113793+0000', 'modified': '2024-09-13T16:53:10.833834+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5983}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5983': {'gid': 5983, 'name': 'c', 'rank': 0, 'incarnation': 34, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.3:6835/2114372521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 2114372521}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 2114372521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:53:11.422 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5983:up:rejoin 2024-09-13T16:53:12.422 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:12.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.611+0000 7ff4cf400700 1 Processor -- start 2024-09-13T16:53:12.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.611+0000 7ff4cf400700 1 -- start start 2024-09-13T16:53:12.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cf400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80cbf50 0x7ff4c806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:12.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cf400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4c8070d60 0x7ff4c8071180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:12.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cf400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c813f6b0 con 0x7ff4c815aab0 2024-09-13T16:53:12.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cf400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c813feb0 con 0x7ff4c8070d60 2024-09-13T16:53:12.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cf400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c80716c0 con 0x7ff4c80cbf50 2024-09-13T16:53:12.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cca00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80cbf50 0x7ff4c806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:12.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4c7e00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff4c815aab0 0x7ff4c813efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:54142/0 (socket says 172.21.15.3:54142) 2024-09-13T16:53:12.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4c7e00700 1 -- 172.21.15.3:0/2802033800 learned_addr learned my addr 172.21.15.3:0/2802033800 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:12.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cd400700 1 --2- 172.21.15.3:0/2802033800 >> v2:172.21.15.157:3300/0 conn(0x7ff4c8070d60 0x7ff4c8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:12.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cca00700 1 -- 172.21.15.3:0/2802033800 >> v2:172.21.15.157:3300/0 conn(0x7ff4c8070d60 msgr2=0x7ff4c8071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:12.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cca00700 1 --2- 172.21.15.3:0/2802033800 >> v2:172.21.15.157:3300/0 conn(0x7ff4c8070d60 0x7ff4c8071180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cca00700 1 -- 172.21.15.3:0/2802033800 >> v1:172.21.15.157:6789/0 conn(0x7ff4c815aab0 legacy=0x7ff4c813efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:12.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4cca00700 1 -- 172.21.15.3:0/2802033800 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c813f8f0 con 0x7ff4c80cbf50 2024-09-13T16:53:12.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.615+0000 7ff4c7400700 1 -- 172.21.15.3:0/2802033800 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318589961 0 0) 0x7ff4b4001110 con 0x7ff4c815aab0 2024-09-13T16:53:12.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cca00700 1 --2- 172.21.15.3:0/2802033800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80cbf50 0x7ff4c806d4a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009850 tx=0x7ff4bc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec087efd1373f061 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:12.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4c7400700 1 -- 172.21.15.3:0/2802033800 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00f8a0 con 0x7ff4c80cbf50 2024-09-13T16:53:12.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4c7400700 1 -- 172.21.15.3:0/2802033800 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc00fa00 con 0x7ff4c80cbf50 2024-09-13T16:53:12.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4c7400700 1 -- 172.21.15.3:0/2802033800 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00fdb0 con 0x7ff4c80cbf50 2024-09-13T16:53:12.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cf400700 1 -- 172.21.15.3:0/2802033800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80cbf50 msgr2=0x7ff4c806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:12.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cf400700 1 --2- 172.21.15.3:0/2802033800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80cbf50 0x7ff4c806d4a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009850 tx=0x7ff4bc009b60 comp rx=0 tx=0).stop 2024-09-13T16:53:12.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cf400700 1 -- 172.21.15.3:0/2802033800 shutdown_connections 2024-09-13T16:53:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cf400700 1 --2- 172.21.15.3:0/2802033800 >> v2:172.21.15.157:3300/0 conn(0x7ff4c8070d60 0x7ff4c8071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cf400700 1 --2- 172.21.15.3:0/2802033800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80cbf50 0x7ff4c806d4a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cf400700 1 -- 172.21.15.3:0/2802033800 >> 172.21.15.3:0/2802033800 conn(0x7ff4c806b400 msgr2=0x7ff4c806b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cf400700 1 -- 172.21.15.3:0/2802033800 shutdown_connections 2024-09-13T16:53:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.619+0000 7ff4cf400700 1 -- 172.21.15.3:0/2802033800 wait complete. 2024-09-13T16:53:12.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 Processor -- start 2024-09-13T16:53:12.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 -- start start 2024-09-13T16:53:12.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8070d60 0x7ff4c8140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:12.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c80cbf50 0x7ff4c8140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:12.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c815aab0 0x7ff4c80ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:12.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8143260 con 0x7ff4c815aab0 2024-09-13T16:53:12.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c815c0b0 con 0x7ff4c80cbf50 2024-09-13T16:53:12.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8142f80 con 0x7ff4c8070d60 2024-09-13T16:53:12.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c815aab0 0x7ff4c80ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:12.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c815aab0 0x7ff4c80ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47670/0 (socket says 172.21.15.3:47670) 2024-09-13T16:53:12.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 -- 172.21.15.3:0/1494623312 learned_addr learned my addr 172.21.15.3:0/1494623312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:12.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cca00700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8070d60 0x7ff4c8140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:12.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4c7e00700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c80cbf50 0x7ff4c8140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:12.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 -- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8070d60 msgr2=0x7ff4c8140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:12.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8070d60 0x7ff4c8140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 -- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c80cbf50 msgr2=0x7ff4c8140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:12.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c80cbf50 0x7ff4c8140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 -- 172.21.15.3:0/1494623312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc0094b0 con 0x7ff4c815aab0 2024-09-13T16:53:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cd400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c815aab0 0x7ff4c80ce090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000efc0 tx=0x7ff4c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4c4c00700 1 -- 172.21.15.3:0/1494623312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0009ea0 con 0x7ff4c815aab0 2024-09-13T16:53:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4c4c00700 1 -- 172.21.15.3:0/1494623312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c000f040 con 0x7ff4c815aab0 2024-09-13T16:53:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c815ba60 con 0x7ff4c815aab0 2024-09-13T16:53:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4c4c00700 1 -- 172.21.15.3:0/1494623312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000ced0 con 0x7ff4c815aab0 2024-09-13T16:53:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.623+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c80cd980 con 0x7ff4c815aab0 2024-09-13T16:53:12.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.627+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c815b8c0 con 0x7ff4c815aab0 2024-09-13T16:53:12.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.627+0000 7ff4c4c00700 1 -- 172.21.15.3:0/1494623312 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff4c0014480 con 0x7ff4c815aab0 2024-09-13T16:53:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.631+0000 7ff4c4c00700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4a8066730 0x7ff4a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:12.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.631+0000 7ff4c4c00700 1 -- 172.21.15.3:0/1494623312 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6201+0+0 (secure 0 0 0) 0x7ff4c0084ec0 con 0x7ff4c815aab0 2024-09-13T16:53:12.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.631+0000 7ff4cca00700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4a8066730 0x7ff4a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:12.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.631+0000 7ff4cca00700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4a8066730 0x7ff4a8068bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ba0 tx=0x7ff4bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:12.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.635+0000 7ff4c4c00700 1 -- 172.21.15.3:0/1494623312 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff4c815b8c0 con 0x7ff4c815aab0 2024-09-13T16:53:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4c815b8c0 con 0x7ff4c815aab0 2024-09-13T16:53:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4c4c00700 1 -- 172.21.15.3:0/1494623312 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7ff4c815b8c0 con 0x7ff4c815aab0 2024-09-13T16:53:12.865 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:12.865 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:12.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4a8066730 msgr2=0x7ff4a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:12.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4a8066730 0x7ff4a8068bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ba0 tx=0x7ff4bc008030 comp rx=0 tx=0).stop 2024-09-13T16:53:12.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c815aab0 msgr2=0x7ff4c80ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c815aab0 0x7ff4c80ce090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000efc0 tx=0x7ff4c000c3a0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 shutdown_connections 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4a8066730 0x7ff4a8068bf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c815aab0 0x7ff4c80ce090 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c80cbf50 0x7ff4c8140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 --2- 172.21.15.3:0/1494623312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8070d60 0x7ff4c8140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.859+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 >> 172.21.15.3:0/1494623312 conn(0x7ff4c806b400 msgr2=0x7ff4c806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.863+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 shutdown_connections 2024-09-13T16:53:12.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:12.863+0000 7ff4cf400700 1 -- 172.21.15.3:0/1494623312 wait complete. 2024-09-13T16:53:12.869 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:13.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:53:00.113793+0000', 'modified': '2024-09-13T16:53:12.847855+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5983}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5983': {'gid': 5983, 'name': 'c', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.3:6835/2114372521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 2114372521}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 2114372521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:53:13.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:53:13.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T16:53:13.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.463+0000 7f9af6000700 1 Processor -- start 2024-09-13T16:53:13.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.463+0000 7f9af6000700 1 -- start start 2024-09-13T16:53:13.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.463+0000 7f9af6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af015cc40 0x7f9af006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.463+0000 7f9af6000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9af006d540 0x7f9af00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.463+0000 7f9af6000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af00adc60 con 0x7f9af015d020 2024-09-13T16:53:13.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.463+0000 7f9af6000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af006da50 con 0x7f9af006d540 2024-09-13T16:53:13.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.463+0000 7f9af6000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af006dd30 con 0x7f9af015cc40 2024-09-13T16:53:13.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af5600700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9af015d020 0x7f9af01609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48798/0 (socket says 172.21.15.3:48798) 2024-09-13T16:53:13.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af5600700 1 -- 172.21.15.3:0/4292902401 learned_addr learned my addr 172.21.15.3:0/4292902401 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:13.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aefe00700 1 --2- 172.21.15.3:0/4292902401 >> v2:172.21.15.157:3300/0 conn(0x7f9af006d540 0x7f9af00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af4c00700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af015cc40 0x7f9af006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942231043 0 0) 0x7f9af00adc60 con 0x7f9af015d020 2024-09-13T16:53:13.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ad80035d0 con 0x7f9af015d020 2024-09-13T16:53:13.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2790256381 0 0) 0x7f9ad80035d0 con 0x7f9af015d020 2024-09-13T16:53:13.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af015cc40 msgr2=0x7f9af006fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af015cc40 0x7f9af006fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 >> v2:172.21.15.157:3300/0 conn(0x7f9af006d540 msgr2=0x7f9af00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 --2- 172.21.15.3:0/4292902401 >> v2:172.21.15.157:3300/0 conn(0x7f9af006d540 0x7f9af00ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00ad610 con 0x7f9af015d020 2024-09-13T16:53:13.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f9ae80019f0 con 0x7f9af015d020 2024-09-13T16:53:13.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 >> v1:172.21.15.3:6789/0 conn(0x7f9af015d020 legacy=0x7f9af01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af015cc40 0x7f9ad8003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8003fa0 0x7f9ad8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad8004db0 0x7f9ad8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af006da50 con 0x7f9ad8004db0 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af006dd30 con 0x7f9ad8003fa0 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad800ba30 con 0x7f9af015cc40 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af5600700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad8004db0 0x7f9ad8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af4c00700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af015cc40 0x7f9ad8003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9aefe00700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8003fa0 0x7f9ad8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af5600700 1 -- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af015cc40 msgr2=0x7f9ad8003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af5600700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af015cc40 0x7f9ad8003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af5600700 1 -- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8003fa0 msgr2=0x7f9ad8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af5600700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8003fa0 0x7f9ad8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.467+0000 7f9af5600700 1 -- 172.21.15.3:0/4292902401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80035d0 con 0x7f9ad8004db0 2024-09-13T16:53:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9af5600700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad8004db0 0x7f9ad8005230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800a860 tx=0x7f9ad80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50927ec64170b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80072d0 con 0x7f9ad8004db0 2024-09-13T16:53:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8007450 con 0x7f9ad8004db0 2024-09-13T16:53:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9aef400700 1 -- 172.21.15.3:0/4292902401 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8007800 con 0x7f9ad8004db0 2024-09-13T16:53:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9af6000700 1 -- 172.21.15.3:0/4292902401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad8004db0 msgr2=0x7f9ad8005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9af6000700 1 --2- 172.21.15.3:0/4292902401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad8004db0 0x7f9ad8005230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800a860 tx=0x7f9ad80058a0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9af5600700 1 -- 172.21.15.3:0/4292902401 reap_dead start 2024-09-13T16:53:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9af6000700 1 -- 172.21.15.3:0/4292902401 shutdown_connections 2024-09-13T16:53:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9af6000700 1 -- 172.21.15.3:0/4292902401 >> 172.21.15.3:0/4292902401 conn(0x7f9af006a2f0 msgr2=0x7f9af006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9af6000700 1 -- 172.21.15.3:0/4292902401 shutdown_connections 2024-09-13T16:53:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.471+0000 7f9af6000700 1 -- 172.21.15.3:0/4292902401 wait complete. 2024-09-13T16:53:13.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 Processor -- start 2024-09-13T16:53:13.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 -- start start 2024-09-13T16:53:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8004db0 0x7f9af015df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af0143120 0x7f9af0159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af01599c0 0x7f9af00cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00cb150 con 0x7f9af0143120 2024-09-13T16:53:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00cb360 con 0x7f9ad8004db0 2024-09-13T16:53:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00cb640 con 0x7f9af01599c0 2024-09-13T16:53:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af0143120 0x7f9af0159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af0143120 0x7f9af0159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47674/0 (socket says 172.21.15.3:47674) 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 -- 172.21.15.3:0/1176344713 learned_addr learned my addr 172.21.15.3:0/1176344713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af5600700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af01599c0 0x7f9af00cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af4c00700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8004db0 0x7f9af015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 -- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af01599c0 msgr2=0x7f9af00cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af01599c0 0x7f9af00cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 -- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8004db0 msgr2=0x7f9af015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8004db0 0x7f9af015df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 -- 172.21.15.3:0/1176344713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00cb920 con 0x7f9af0143120 2024-09-13T16:53:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aefe00700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af0143120 0x7f9af0159480 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006df0 tx=0x7f9adc011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aecc00700 1 -- 172.21.15.3:0/1176344713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc004030 con 0x7f9af0143120 2024-09-13T16:53:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aecc00700 1 -- 172.21.15.3:0/1176344713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc004190 con 0x7f9af0143120 2024-09-13T16:53:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9aecc00700 1 -- 172.21.15.3:0/1176344713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc004540 con 0x7f9af0143120 2024-09-13T16:53:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.475+0000 7f9af6000700 1 -- 172.21.15.3:0/1176344713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af00cbc00 con 0x7f9af0143120 2024-09-13T16:53:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.479+0000 7f9af6000700 1 -- 172.21.15.3:0/1176344713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af01435a0 con 0x7f9af0143120 2024-09-13T16:53:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.483+0000 7f9aecc00700 1 -- 172.21.15.3:0/1176344713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9adc00f940 con 0x7f9af0143120 2024-09-13T16:53:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.483+0000 7f9aecc00700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ad0066760 0x7f9ad0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.483+0000 7f9aecc00700 1 -- 172.21.15.3:0/1176344713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f9adc0847b0 con 0x7f9af0143120 2024-09-13T16:53:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.483+0000 7f9af4c00700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ad0066760 0x7f9ad0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9afaab9740 1 Processor -- start 2024-09-13T16:53:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af4c00700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ad0066760 0x7f9ad0068c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9af00b2b20 tx=0x7f9ae4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9afaab9740 1 -- start start 2024-09-13T16:53:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3da0c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3d9d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9afaab9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x3d933d80 con 0x3da0c6c0 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9afaab9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x3d931510 con 0x3da0bda0 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9afaab9740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3d92ce00 con 0x3d9ead00 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3d9d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3d9d4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47690/0 (socket says 172.21.15.3:47690) 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 -- 172.21.15.3:0/573107567 learned_addr learned my addr 172.21.15.3:0/573107567 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9aefe00700 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af4c00700 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3da0c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 -- 172.21.15.3:0/573107567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 msgr2=0x3d9d4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9d4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 -- 172.21.15.3:0/573107567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 msgr2=0x3da0c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3da0c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 -- 172.21.15.3:0/573107567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x3d79f960 con 0x3da0c6c0 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9af5600700 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3d9d4dd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8000f80 tx=0x7f9ae8006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fa204c02d68ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9ae1200700 1 -- 172.21.15.3:0/573107567 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80019b0 con 0x3da0c6c0 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9ae1200700 1 -- 172.21.15.3:0/573107567 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae800aec0 con 0x3da0c6c0 2024-09-13T16:53:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.487+0000 7f9ae1200700 1 -- 172.21.15.3:0/573107567 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8007250 con 0x3da0c6c0 2024-09-13T16:53:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- 172.21.15.3:0/573107567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 msgr2=0x3d9d4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3d9d4dd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8000f80 tx=0x7f9ae8006000 comp rx=0 tx=0).stop 2024-09-13T16:53:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- 172.21.15.3:0/573107567 shutdown_connections 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3d9d4dd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9d4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 --2- 172.21.15.3:0/573107567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3da0c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- 172.21.15.3:0/573107567 >> 172.21.15.3:0/573107567 conn(0x3d7544d0 msgr2=0x3d9edc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- 172.21.15.3:0/573107567 shutdown_connections 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- 172.21.15.3:0/573107567 wait complete. 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- start start 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 Processor -- start 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3d9ead00 0x3da0c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3da0bda0 0x3da12340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0c6c0 0x3da07480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x3d8ea730 con 0x3d9ead00 2024-09-13T16:53:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x3d8f05b0 con 0x3da0c6c0 2024-09-13T16:53:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3d7d8aa0 con 0x3da0bda0 2024-09-13T16:53:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9afaab9740 10 client.0 _ll_register_callbacks cb 0x3d926610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:53:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9af4c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3d9ead00 0x3da0c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9af4c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3d9ead00 0x3da0c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47704/0 (socket says 172.21.15.3:47704) 2024-09-13T16:53:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.491+0000 7f9af4c00700 1 -- 172.21.15.3:0/2656377331 learned_addr learned my addr 172.21.15.3:0/2656377331 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9aefe00700 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3da0bda0 0x3da12340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9af5600700 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0c6c0 0x3da07480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9af4c00700 1 -- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3da0bda0 msgr2=0x3da12340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9af4c00700 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3da0bda0 0x3da12340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9af4c00700 1 -- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0c6c0 msgr2=0x3da07480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9af4c00700 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0c6c0 0x3da07480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9af4c00700 1 -- 172.21.15.3:0/2656377331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad80035d0 con 0x3d9ead00 2024-09-13T16:53:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9af5600700 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0c6c0 0x3da07480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9af4c00700 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3d9ead00 0x3da0c180 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40169b0 tx=0x7f9ae40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da88ca792b3631c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9ad5600700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9ad5600700 1 -- 172.21.15.3:0/2656377331 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4023850 con 0x3d9ead00 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9ad5600700 1 -- 172.21.15.3:0/2656377331 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f9ae40239b0 con 0x3d9ead00 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 1 -- 172.21.15.3:0/2656377331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x3d7ab460 con 0x3d9ead00 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9ad5600700 1 -- 172.21.15.3:0/2656377331 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ae4023b70 con 0x3d9ead00 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 10 client.6039 fetch_fsmap learned FSMap version 38 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 1 -- 172.21.15.3:0/2656377331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x3d8f3220 con 0x3d9ead00 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9ad5600700 1 -- 172.21.15.3:0/2656377331 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x3d7ab460 con 0x3d9ead00 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9ad5600700 1 -- 172.21.15.3:0/2656377331 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9ae4023cd0 con 0x3d9ead00 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 10 client.6039 fetch_fsmap finished waiting for FSMap version 38 2024-09-13T16:53:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 10 client.6039 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T16:53:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 20 client.6039 populate_metadata read hostname 'smithi003' 2024-09-13T16:53:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] conn(0x3d9c45f0 0x3d9c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 4 client.6039 mds_command: new command op to 5983 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:53:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9afaab9740 1 -- 172.21.15.3:0/2656377331 --> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x3d7d01e0 con 0x3d9c45f0 2024-09-13T16:53:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9aefe00700 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] conn(0x3d9c45f0 0x3d9c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9ad5600700 10 client.6039 ms_handle_connect on v2:172.21.15.3:6834/2114372521 2024-09-13T16:53:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.495+0000 7f9aefe00700 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] conn(0x3d9c45f0 0x3d9c5660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3abcefbda9894ed server_cookie=377f3def277a31f5 in_seq=0 out_seq=0 2024-09-13T16:53:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9ad5600700 1 -- 172.21.15.3:0/2656377331 <== mds.0 v2:172.21.15.3:6834/2114372521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x3d7d01e0 con 0x3d9c45f0 2024-09-13T16:53:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9ad5600700 10 client.6039 handle_command_reply: tid=0 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 client.6039 shutdown 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 -- 172.21.15.3:0/2656377331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3d9ead00 msgr2=0x3da0c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3d9ead00 0x3da0c180 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40169b0 tx=0x7f9ae40047e0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 -- 172.21.15.3:0/2656377331 shutdown_connections 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] conn(0x3d9c45f0 0x3d9c5660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0c6c0 0x3da07480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3da0bda0 0x3da12340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9ad5600700 0 client.6039 ms_handle_reset on v2:172.21.15.3:6834/2114372521 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 --2- 172.21.15.3:0/2656377331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3d9ead00 0x3da0c180 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 -- 172.21.15.3:0/2656377331 >> 172.21.15.3:0/2656377331 conn(0x3d7544d0 msgr2=0x3d9e7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 -- 172.21.15.3:0/2656377331 shutdown_connections 2024-09-13T16:53:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 1 -- 172.21.15.3:0/2656377331 wait complete. 2024-09-13T16:53:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.499+0000 7f9afaab9740 20 client.6039 trim_cache size 0 max 16384 2024-09-13T16:53:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.519+0000 7f9afaab9740 1 Processor -- start 2024-09-13T16:53:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.519+0000 7f9afaab9740 1 -- start start 2024-09-13T16:53:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3d9c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da0caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9afaab9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x3d92bb20 con 0x3da0c6c0 2024-09-13T16:53:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9afaab9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x3d8fb5b0 con 0x3da0bda0 2024-09-13T16:53:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9afaab9740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3d95bec0 con 0x3d9ead00 2024-09-13T16:53:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da0caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da0caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47720/0 (socket says 172.21.15.3:47720) 2024-09-13T16:53:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 -- 172.21.15.3:0/1038013968 learned_addr learned my addr 172.21.15.3:0/1038013968 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9aefe00700 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3d9c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af4c00700 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 -- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 msgr2=0x3d9c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 -- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 msgr2=0x3d9c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3d9c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 -- 172.21.15.3:0/1038013968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x3d8d9450 con 0x3da0c6c0 2024-09-13T16:53:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9aefe00700 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3d9c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af4c00700 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9af5600700 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da0caa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800aca0 tx=0x7f9ae800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c457e295ba5d599a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9ae1c00700 1 -- 172.21.15.3:0/1038013968 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8007030 con 0x3da0c6c0 2024-09-13T16:53:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9ae1c00700 1 -- 172.21.15.3:0/1038013968 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae80071b0 con 0x3da0c6c0 2024-09-13T16:53:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9ae1c00700 1 -- 172.21.15.3:0/1038013968 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80019f0 con 0x3da0c6c0 2024-09-13T16:53:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9afaab9740 1 -- 172.21.15.3:0/1038013968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 msgr2=0x3da0caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.523+0000 7f9afaab9740 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da0caa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800aca0 tx=0x7f9ae800a060 comp rx=0 tx=0).stop 2024-09-13T16:53:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 -- 172.21.15.3:0/1038013968 shutdown_connections 2024-09-13T16:53:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da0caa0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3d9c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 --2- 172.21.15.3:0/1038013968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3d9c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 -- 172.21.15.3:0/1038013968 >> 172.21.15.3:0/1038013968 conn(0x3d7544d0 msgr2=0x3da1c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 -- 172.21.15.3:0/1038013968 shutdown_connections 2024-09-13T16:53:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 -- 172.21.15.3:0/1038013968 wait complete. 2024-09-13T16:53:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 -- start start 2024-09-13T16:53:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 Processor -- start 2024-09-13T16:53:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3da116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3da6d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da6d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x3d9c3fc0 con 0x3da0c6c0 2024-09-13T16:53:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x3d9c3720 con 0x3da0bda0 2024-09-13T16:53:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x3d8f44c0 con 0x3d9ead00 2024-09-13T16:53:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.527+0000 7f9afaab9740 10 client.0 _ll_register_callbacks cb 0x3d983710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:53:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da6d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da6d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47728/0 (socket says 172.21.15.3:47728) 2024-09-13T16:53:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 -- 172.21.15.3:0/3950826371 learned_addr learned my addr 172.21.15.3:0/3950826371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9aefe00700 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3da6d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af4c00700 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3da116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 -- 172.21.15.3:0/3950826371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 msgr2=0x3da116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3da116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 -- 172.21.15.3:0/3950826371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 msgr2=0x3da6d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3da6d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 -- 172.21.15.3:0/3950826371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x3d79f960 con 0x3da0c6c0 2024-09-13T16:53:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af5600700 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da6d570 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8002110 tx=0x7f9ae8001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3d56f88d3eb923 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9ad5600700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-13T16:53:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9ad5600700 1 -- 172.21.15.3:0/3950826371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80019b0 con 0x3da0c6c0 2024-09-13T16:53:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 1 -- 172.21.15.3:0/3950826371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x3d7611e0 con 0x3da0c6c0 2024-09-13T16:53:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9ad5600700 1 -- 172.21.15.3:0/3950826371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f9ae8024020 con 0x3da0c6c0 2024-09-13T16:53:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9ad5600700 1 -- 172.21.15.3:0/3950826371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ae80075c0 con 0x3da0c6c0 2024-09-13T16:53:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 10 client.6045 fetch_fsmap learned FSMap version 38 2024-09-13T16:53:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 1 -- 172.21.15.3:0/3950826371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x3d8cd720 con 0x3da0c6c0 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9ad5600700 1 -- 172.21.15.3:0/3950826371 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x3d7611e0 con 0x3da0c6c0 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9ad5600700 1 -- 172.21.15.3:0/3950826371 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9ae8007720 con 0x3da0c6c0 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 10 client.6045 fetch_fsmap finished waiting for FSMap version 38 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 10 client.6045 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 20 client.6045 populate_metadata read hostname 'smithi003' 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] conn(0x3da6dbd0 0x3da6dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 4 client.6045 mds_command: new command op to 5983 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9afaab9740 1 -- 172.21.15.3:0/3950826371 --> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] -- command(tid 0: {"prefix": "status"}) v1 -- 0x3da6e4f0 con 0x3da6dbd0 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.531+0000 7f9af4c00700 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] conn(0x3da6dbd0 0x3da6dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af4c00700 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] conn(0x3da6dbd0 0x3da6dfb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12caa7a0a25bead2 server_cookie=e953a825c13d8ccf in_seq=0 out_seq=0 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9ad5600700 10 client.6045 ms_handle_connect on v2:172.21.15.3:6834/2114372521 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9ad5600700 1 -- 172.21.15.3:0/3950826371 <== mds.0 v2:172.21.15.3:6834/2114372521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x3da6e4f0 con 0x3da6dbd0 2024-09-13T16:53:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9ad5600700 10 client.6045 handle_command_reply: tid=0 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 client.6045 shutdown 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 -- 172.21.15.3:0/3950826371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 msgr2=0x3da6d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da6d570 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8002110 tx=0x7f9ae8001df0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 -- 172.21.15.3:0/3950826371 shutdown_connections 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x3da0c6c0 0x3da6d570 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.3:6834/2114372521,v1:172.21.15.3:6835/2114372521] conn(0x3da6dbd0 0x3da6dfb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x3da0bda0 0x3da6d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 --2- 172.21.15.3:0/3950826371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x3d9ead00 0x3da116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 -- 172.21.15.3:0/3950826371 >> 172.21.15.3:0/3950826371 conn(0x3d7544d0 msgr2=0x3da0a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9ad5600700 0 client.6045 ms_handle_reset on v2:172.21.15.3:6834/2114372521 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 -- 172.21.15.3:0/3950826371 shutdown_connections 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 1 -- 172.21.15.3:0/3950826371 wait complete. 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9afaab9740 20 client.6045 trim_cache size 0 max 16384 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "61456116-a42d-4065-9bf5-a1daf0be594e", 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5983, 2024-09-13T16:53:13.546 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-13T16:53:13.547 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-13T16:53:13.547 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-13T16:53:13.547 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 4.7146417319999996, 2024-09-13T16:53:13.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 38, 2024-09-13T16:53:13.547 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 62, 2024-09-13T16:53:13.547 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 62, 2024-09-13T16:53:13.547 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 4.7422797450000003 2024-09-13T16:53:13.547 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:53:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 -- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ad0066760 msgr2=0x7f9ad0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ad0066760 0x7f9ad0068c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9af00b2b20 tx=0x7f9ae4008040 comp rx=0 tx=0).stop 2024-09-13T16:53:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 -- 172.21.15.3:0/1176344713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af0143120 msgr2=0x7f9af0159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af0143120 0x7f9af0159480 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006df0 tx=0x7f9adc011e00 comp rx=0 tx=0).stop 2024-09-13T16:53:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 -- 172.21.15.3:0/1176344713 shutdown_connections 2024-09-13T16:53:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9ad0066760 0x7f9ad0068c20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9af01599c0 0x7f9af00cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af0143120 0x7f9af0159480 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 --2- 172.21.15.3:0/1176344713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8004db0 0x7f9af015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 -- 172.21.15.3:0/1176344713 >> 172.21.15.3:0/1176344713 conn(0x7f9af006a2f0 msgr2=0x7f9af014dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 -- 172.21.15.3:0/1176344713 shutdown_connections 2024-09-13T16:53:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.535+0000 7f9af6000700 1 -- 172.21.15.3:0/1176344713 wait complete. 2024-09-13T16:53:13.566 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:13.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.731+0000 7f6cd3a00700 1 Processor -- start 2024-09-13T16:53:13.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.731+0000 7f6cd3a00700 1 -- start start 2024-09-13T16:53:13.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6cd3a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ccc0b42d0 0x7f6ccc0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6cd3a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc0b4c70 con 0x7f6ccc15c160 2024-09-13T16:53:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6cd3a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc0cc070 con 0x7f6ccc15def0 2024-09-13T16:53:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6cd3a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc0cccf0 con 0x7f6ccc0b42d0 2024-09-13T16:53:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6cd1a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ccc15c160 0x7f6ccc15c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56110/0 (socket says 172.21.15.3:56110) 2024-09-13T16:53:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6cd1a00700 1 -- 172.21.15.3:0/1392471541 learned_addr learned my addr 172.21.15.3:0/1392471541 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6ccbe00700 1 --2- 172.21.15.3:0/1392471541 >> v2:172.21.15.157:3300/0 conn(0x7f6ccc0b42d0 0x7f6ccc0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796277913 0 0) 0x7f6ccc0b4c70 con 0x7f6ccc15c160 2024-09-13T16:53:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb40035d0 con 0x7f6ccc15c160 2024-09-13T16:53:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267410016 0 0) 0x7f6ccc0cc070 con 0x7f6ccc15def0 2024-09-13T16:53:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ccc0b4c70 con 0x7f6ccc15def0 2024-09-13T16:53:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.735+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174211033 0 0) 0x7f6cb40035d0 con 0x7f6ccc15c160 2024-09-13T16:53:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 >> v2:172.21.15.157:3300/0 conn(0x7f6ccc0b42d0 msgr2=0x7f6ccc0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 --2- 172.21.15.3:0/1392471541 >> v2:172.21.15.157:3300/0 conn(0x7f6ccc0b42d0 0x7f6ccc0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 >> v1:172.21.15.157:6789/0 conn(0x7f6ccc15def0 legacy=0x7f6ccc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0cc350 con 0x7f6ccc15c160 2024-09-13T16:53:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f6cc40019f0 con 0x7f6ccc15c160 2024-09-13T16:53:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 >> v1:172.21.15.3:6789/0 conn(0x7f6ccc15c160 legacy=0x7f6ccc15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4003bb0 0x7f6cb4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4004870 0x7f6cb4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb4005230 0x7f6cb400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0cccf0 con 0x7f6cb4005230 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb400c160 con 0x7f6cb4003bb0 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb400c320 con 0x7f6cb4004870 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccbe00700 1 --2- 172.21.15.3:0/1392471541 >> v2:172.21.15.157:3300/0 conn(0x7f6ccc0b42d0 0x7f6ccc0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1a00700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb4005230 0x7f6cb400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1000700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4003bb0 0x7f6cb4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccbe00700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4004870 0x7f6cb4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1a00700 1 -- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4004870 msgr2=0x7f6cb4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1a00700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4004870 0x7f6cb4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1a00700 1 -- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4003bb0 msgr2=0x7f6cb4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1a00700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4003bb0 0x7f6cb4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1a00700 1 -- 172.21.15.3:0/1392471541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40035d0 con 0x7f6cb4005230 2024-09-13T16:53:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1000700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4003bb0 0x7f6cb4003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1a00700 1 -- 172.21.15.3:0/1392471541 reap_dead start 2024-09-13T16:53:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd1a00700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb4005230 0x7f6cb400ba30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400a7c0 tx=0x7f6cb40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87513fa83111b4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40072d0 con 0x7f6cb4005230 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc4007450 con 0x7f6cb4005230 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6ccb400700 1 -- 172.21.15.3:0/1392471541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4007800 con 0x7f6cb4005230 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd3a00700 1 -- 172.21.15.3:0/1392471541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb4005230 msgr2=0x7f6cb400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.739+0000 7f6cd3a00700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb4005230 0x7f6cb400ba30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400a7c0 tx=0x7f6cb40058a0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.743+0000 7f6cd3a00700 1 -- 172.21.15.3:0/1392471541 shutdown_connections 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.743+0000 7f6cd3a00700 1 --2- 172.21.15.3:0/1392471541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb4005230 0x7f6cb400ba30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.743+0000 7f6cd3a00700 1 -- 172.21.15.3:0/1392471541 >> 172.21.15.3:0/1392471541 conn(0x7f6ccc0b0cc0 msgr2=0x7f6ccc0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.743+0000 7f6cd3a00700 1 -- 172.21.15.3:0/1392471541 shutdown_connections 2024-09-13T16:53:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.743+0000 7f6cd3a00700 1 -- 172.21.15.3:0/1392471541 wait complete. 2024-09-13T16:53:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.743+0000 7f6cd3a00700 1 Processor -- start 2024-09-13T16:53:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.743+0000 7f6cd3a00700 1 -- start start 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd3a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4005230 0x7f6ccc1520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd3a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ccc13a130 0x7f6ccc140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd3a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ccc13dad0 0x7f6ccc13ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd3a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc152600 con 0x7f6ccc13a130 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd3a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc141100 con 0x7f6ccc13dad0 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd3a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc1527f0 con 0x7f6cb4005230 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ccc13a130 0x7f6ccc140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd1000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4005230 0x7f6ccc1520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ccc13a130 0x7f6ccc140bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55782/0 (socket says 172.21.15.3:55782) 2024-09-13T16:53:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 -- 172.21.15.3:0/3450346546 learned_addr learned my addr 172.21.15.3:0/3450346546 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:13.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd1a00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ccc13dad0 0x7f6ccc13ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 -- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4005230 msgr2=0x7f6ccc1520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4005230 0x7f6ccc1520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 -- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ccc13dad0 msgr2=0x7f6ccc13ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ccc13dad0 0x7f6ccc13ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 -- 172.21.15.3:0/3450346546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc152ad0 con 0x7f6ccc13a130 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6ccbe00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ccc13a130 0x7f6ccc140bc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80109e0 tx=0x7f6cb8010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cc8c00700 1 -- 172.21.15.3:0/3450346546 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8010e80 con 0x7f6ccc13a130 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cc8c00700 1 -- 172.21.15.3:0/3450346546 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb800e8d0 con 0x7f6ccc13a130 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc152db0 con 0x7f6ccc13a130 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cc8c00700 1 -- 172.21.15.3:0/3450346546 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800ec40 con 0x7f6ccc13a130 2024-09-13T16:53:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.747+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc153370 con 0x7f6ccc13a130 2024-09-13T16:53:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.751+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc15bdd0 con 0x7f6ccc13a130 2024-09-13T16:53:13.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.751+0000 7f6cc8c00700 1 -- 172.21.15.3:0/3450346546 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6cb8005a40 con 0x7f6ccc13a130 2024-09-13T16:53:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.755+0000 7f6cc8c00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6cac066760 0x7f6cac068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.755+0000 7f6cc8c00700 1 -- 172.21.15.3:0/3450346546 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f6cb8084390 con 0x7f6ccc13a130 2024-09-13T16:53:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.755+0000 7f6cd1000700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6cac066760 0x7f6cac068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.755+0000 7f6cd1000700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6cac066760 0x7f6cac068c20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0007fd0 tx=0x7f6cc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.759+0000 7f6cc8c00700 1 -- 172.21.15.3:0/3450346546 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6cb8051470 con 0x7f6ccc13a130 2024-09-13T16:53:13.834 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-13T16:53:13.827+0000 7fd276200700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:13.999+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ccc15aae0 con 0x7f6ccc13a130 2024-09-13T16:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cc8c00700 1 -- 172.21.15.3:0/3450346546 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f6cb8054a90 con 0x7f6ccc13a130 2024-09-13T16:53:14.008 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:14.008 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6cac066760 msgr2=0x7f6cac068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6cac066760 0x7f6cac068c20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0007fd0 tx=0x7f6cc0009040 comp rx=0 tx=0).stop 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ccc13a130 msgr2=0x7f6ccc140bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ccc13a130 0x7f6ccc140bc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80109e0 tx=0x7f6cb8010cf0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 shutdown_connections 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6cac066760 0x7f6cac068c20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ccc13dad0 0x7f6ccc13ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ccc13a130 0x7f6ccc140bc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 --2- 172.21.15.3:0/3450346546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4005230 0x7f6ccc1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.003+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 >> 172.21.15.3:0/3450346546 conn(0x7f6ccc0b0cc0 msgr2=0x7f6ccc15afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.007+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 shutdown_connections 2024-09-13T16:53:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:14.007+0000 7f6cd3a00700 1 -- 172.21.15.3:0/3450346546 wait complete. 2024-09-13T16:53:14.013 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:14.033 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":23,"msgr_send_messages":23,"msgr_recv_bytes":13774,"msgr_send_bytes":25382,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.015406668,"msgr_running_send_time":0.001696898,"msgr_running_recv_time":0.006925448,"msgr_running_fast_dispatch_time":0.000667128,"msgr_send_messages_queue_lat":{"avgcount":22,"sum":0.009788936,"avgtime":0.000444951},"msgr_handle_ack_lat":{"avgcount":8,"sum":0.000002561,"avgtime":0.000000320},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":45,"msgr_send_messages":40,"msgr_recv_bytes":116704,"msgr_send_bytes":15556,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.012675221,"msgr_running_send_time":0.002364001,"msgr_running_recv_time":0.015504799,"msgr_running_fast_dispatch_time":0.000941484,"msgr_send_messages_queue_lat":{"avgcount":40,"sum":0.010660482,"avgtime":0.000266512},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":102704,"msgr_send_encrypted_bytes":5280},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":12,"msgr_recv_bytes":3670,"msgr_send_bytes":61556,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.005912886,"msgr_running_send_time":0.000899490,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":11,"sum":0.005451806,"avgtime":0.000495618},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":58880},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":48,"sum":0.006118761,"avgtime":0.000127474}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":2,"reply":0,"reply_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":7,"inodes_bottom":3,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":1,"traverse_hit":1,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":0},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":2,"evex":0,"evtrm":0,"ev":4,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4196386,"rdpos":4195196,"jlat":{"avgcount":2,"sum":0.005739088,"avgtime":0.002869544},"replayed":2},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":1,"cap-":0,"rss":45656,"heap":223608},"mds_server":{"dispatch_client_request":1,"dispatch_server_request":0,"handle_client_request":2,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":0,"average_load":0,"avg_session_uptime":951,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238382,"buffer_anon_items":72,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5784,"osdmap_items":93,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51904,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":46,"op_send_bytes":1469,"op_resend":0,"op_reply":46,"op_latency":{"avgcount":46,"sum":0.103998186,"avgtime":0.002260830},"op_inflight":0,"oplen_avg":{"avgcount":46,"sum":92},"op":46,"op_r":36,"op_w":10,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":2,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":1,"omap_rd":48,"omap_del":0},"oft":{"omap_total_objs":0,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":72,"get_sum":125126,"get_or_fail_fail":0,"get_or_fail_success":72,"take":0,"take_sum":0,"put":72,"put_sum":125126,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":2356,"put":46,"put_sum":2356,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":46,"put":46,"put_sum":46,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":2,"get_sum":931,"get_or_fail_fail":0,"get_or_fail_success":2,"take":0,"take_sum":0,"put":2,"put_sum":931,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:14.219 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 8, "avgtime": 3.2e-07, "sum": 2.561e-06 }, "msgr_recv_bytes": 13774, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.000667128, "msgr_running_recv_time": 0.006925448, "msgr_running_send_time": 0.001696898, "msgr_running_total_time": 0.015406668, "msgr_send_bytes": 25382, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 23, "msgr_send_messages_queue_lat": { "avgcount": 22, "avgtime": 0.000444951, "sum": 0.009788936 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 116704, "msgr_recv_encrypted_bytes": 102704, "msgr_recv_messages": 45, "msgr_running_fast_dispatch_time": 0.000941484, "msgr_running_recv_time": 0.015504799, "msgr_running_send_time": 0.002364001, "msgr_running_total_time": 0.012675221, "msgr_send_bytes": 15556, "msgr_send_encrypted_bytes": 5280, "msgr_send_messages": 40, "msgr_send_messages_queue_lat": { "avgcount": 40, "avgtime": 0.000266512, "sum": 0.010660482 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.00089949, "msgr_running_total_time": 0.005912886, "msgr_send_bytes": 61556, "msgr_send_encrypted_bytes": 58880, "msgr_send_messages": 12, "msgr_send_messages_queue_lat": { "avgcount": 11, "avgtime": 0.000495618, "sum": 0.005451806 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 48, "avgtime": 0.000127474, "sum": 0.006118761 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 3, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 7, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 0, "q": 0, "reply": 0, "reply_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "request": 2, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 1, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 1, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 4, "evadd": 2, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 2, "avgtime": 0.002869544, "sum": 0.005739088 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4196386 }, "mds_mem": { "cap": 1, "cap+": 1, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 223608, "ino": 13, "ino+": 13, "ino-": 0, "rss": 45656 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 1, "dispatch_server_request": 0, "handle_client_request": 2, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 0, "avg_session_uptime": 951, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 0 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238382, "buffer_anon_items": 72, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51904, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5784, "osdmap_items": 93, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 0, "omap_rd": 48, "omap_wr": 1, "op": 46, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 46, "avgtime": 0.00226083, "sum": 0.103998186 }, "op_pg": 0, "op_r": 36, "op_reply": 46, "op_resend": 0, "op_rmw": 0, "op_send": 46, "op_send_bytes": 1469, "op_w": 10, "oplen_avg": { "avgcount": 46, "sum": 92 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 0, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 72, "get_or_fail_fail": 0, "get_or_fail_success": 72, "get_started": 0, "get_sum": 125126, "max": 104857600, "put": 72, "put_sum": 125126, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 46, "put_sum": 2356, "take": 46, "take_sum": 2356, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 46, "put_sum": 46, "take": 46, "take_sum": 46, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 2, "get_or_fail_fail": 0, "get_or_fail_success": 2, "get_started": 0, "get_sum": 931, "max": 3758096384, "put": 2, "put_sum": 931, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:14.219 INFO:tasks.cephfs.test_openfiletable:omap_total_updates_0:0 2024-09-13T16:53:14.219 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:53:14.219 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:53:14.235 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:53:14.236 DEBUG:teuthology.orchestra.run.smithi003:> import time 2024-09-13T16:53:14.236 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-13T16:53:14.236 DEBUG:teuthology.orchestra.run.smithi003:> with open("/home/ubuntu/cephtest/mnt.0/omap_counter_test_file", '"'"'w'"'"') as f: 2024-09-13T16:53:14.236 DEBUG:teuthology.orchestra.run.smithi003:> f.write("content") 2024-09-13T16:53:14.236 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2024-09-13T16:53:14.237 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2024-09-13T16:53:14.237 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2024-09-13T16:53:14.237 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-13T16:53:14.285 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/omap_counter_test_file 2024-09-13T16:53:14.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:53:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/omap_counter_test_file': No such file or directory 2024-09-13T16:53:15.304 DEBUG:teuthology.orchestra.run.smithi003:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/omap_counter_test_file 2024-09-13T16:53:15.321 INFO:teuthology.orchestra.run.smithi003.stdout:7 2024-09-13T16:53:15.321 INFO:tasks.cephfs.mount:File omap_counter_test_file became visible with size 7 from 0 after 1s 2024-09-13T16:53:15.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.531+0000 7f44e0a00700 1 Processor -- start 2024-09-13T16:53:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.535+0000 7f44e0a00700 1 -- start start 2024-09-13T16:53:15.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.535+0000 7f44e0a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44dc071810 0x7f44dc0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.535+0000 7f44e0a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44dc070c70 con 0x7f44dc15cb40 2024-09-13T16:53:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.535+0000 7f44e0a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44dc15cf20 con 0x7f44dc0cf3b0 2024-09-13T16:53:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.535+0000 7f44e0a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc0acc20 con 0x7f44dc071810 2024-09-13T16:53:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.535+0000 7f44db400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44dc15cb40 0x7f44dc160690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56114/0 (socket says 172.21.15.3:56114) 2024-09-13T16:53:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.535+0000 7f44db400700 1 -- 172.21.15.3:0/2598130987 learned_addr learned my addr 172.21.15.3:0/2598130987 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.535+0000 7f44dbe00700 1 --2- 172.21.15.3:0/2598130987 >> v2:172.21.15.157:3300/0 conn(0x7f44dc071810 0x7f44dc0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44da000700 1 -- 172.21.15.3:0/2598130987 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649029543 0 0) 0x7f44dc070c70 con 0x7f44dc15cb40 2024-09-13T16:53:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44da000700 1 -- 172.21.15.3:0/2598130987 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44c00035d0 con 0x7f44dc15cb40 2024-09-13T16:53:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44da000700 1 -- 172.21.15.3:0/2598130987 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004947600 0 0) 0x7f44dc15cf20 con 0x7f44dc0cf3b0 2024-09-13T16:53:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44da000700 1 -- 172.21.15.3:0/2598130987 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44dc070c70 con 0x7f44dc0cf3b0 2024-09-13T16:53:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44dbe00700 1 -- 172.21.15.3:0/2598130987 >> v1:172.21.15.157:6789/0 conn(0x7f44dc0cf3b0 legacy=0x7f44dc071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44dbe00700 1 -- 172.21.15.3:0/2598130987 >> v1:172.21.15.3:6789/0 conn(0x7f44dc15cb40 legacy=0x7f44dc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44dbe00700 1 -- 172.21.15.3:0/2598130987 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc0acf00 con 0x7f44dc071810 2024-09-13T16:53:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44dbe00700 1 --2- 172.21.15.3:0/2598130987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc071810 0x7f44dc0aeb20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009d90 tx=0x7f44d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147cb0983a696c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:15.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44da000700 1 -- 172.21.15.3:0/2598130987 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d400e070 con 0x7f44dc071810 2024-09-13T16:53:15.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44da000700 1 -- 172.21.15.3:0/2598130987 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d400bb20 con 0x7f44dc071810 2024-09-13T16:53:15.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44da000700 1 -- 172.21.15.3:0/2598130987 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4012430 con 0x7f44dc071810 2024-09-13T16:53:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44e0a00700 1 -- 172.21.15.3:0/2598130987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc071810 msgr2=0x7f44dc0aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44e0a00700 1 --2- 172.21.15.3:0/2598130987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc071810 0x7f44dc0aeb20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009d90 tx=0x7f44d400b330 comp rx=0 tx=0).stop 2024-09-13T16:53:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44e0a00700 1 -- 172.21.15.3:0/2598130987 shutdown_connections 2024-09-13T16:53:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44e0a00700 1 --2- 172.21.15.3:0/2598130987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc071810 0x7f44dc0aeb20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:15.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.539+0000 7f44e0a00700 1 -- 172.21.15.3:0/2598130987 >> 172.21.15.3:0/2598130987 conn(0x7f44dc06b400 msgr2=0x7f44dc06b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:15.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 -- 172.21.15.3:0/2598130987 shutdown_connections 2024-09-13T16:53:15.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 -- 172.21.15.3:0/2598130987 wait complete. 2024-09-13T16:53:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 Processor -- start 2024-09-13T16:53:15.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 -- start start 2024-09-13T16:53:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44dc071810 0x7f44dc1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc0cf3b0 0x7f44dc15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44dc15cb40 0x7f44dc15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc15b220 con 0x7f44dc071810 2024-09-13T16:53:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c6190 con 0x7f44dc0cf3b0 2024-09-13T16:53:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44e0a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc15ab60 con 0x7f44dc15cb40 2024-09-13T16:53:15.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44db400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44dc071810 0x7f44dc1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.543+0000 7f44db400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44dc071810 0x7f44dc1574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55786/0 (socket says 172.21.15.3:55786) 2024-09-13T16:53:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44db400700 1 -- 172.21.15.3:0/2106641660 learned_addr learned my addr 172.21.15.3:0/2106641660 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44dbe00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44dc15cb40 0x7f44dc15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44daa00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc0cf3b0 0x7f44dc15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44db400700 1 -- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44dc15cb40 msgr2=0x7f44dc15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44db400700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44dc15cb40 0x7f44dc15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44db400700 1 -- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc0cf3b0 msgr2=0x7f44dc15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44db400700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc0cf3b0 0x7f44dc15b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44db400700 1 -- 172.21.15.3:0/2106641660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4009a70 con 0x7f44dc071810 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44db400700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44dc071810 0x7f44dc1574a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c580 tx=0x7f44cc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44d3400700 1 -- 172.21.15.3:0/2106641660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc005d90 con 0x7f44dc071810 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44d3400700 1 -- 172.21.15.3:0/2106641660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc0070d0 con 0x7f44dc071810 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44d3400700 1 -- 172.21.15.3:0/2106641660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc007460 con 0x7f44dc071810 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc15ae40 con 0x7f44dc071810 2024-09-13T16:53:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc1579e0 con 0x7f44dc071810 2024-09-13T16:53:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.547+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc15b360 con 0x7f44dc071810 2024-09-13T16:53:15.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.551+0000 7f44d3400700 1 -- 172.21.15.3:0/2106641660 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44cc0075c0 con 0x7f44dc071810 2024-09-13T16:53:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.551+0000 7f44d3400700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f44bc066730 0x7f44bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.551+0000 7f44d3400700 1 -- 172.21.15.3:0/2106641660 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f44cc083900 con 0x7f44dc071810 2024-09-13T16:53:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.551+0000 7f44daa00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f44bc066730 0x7f44bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:15.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.555+0000 7f44daa00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f44bc066730 0x7f44bc068bf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f44c8007fd0 tx=0x7f44c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.555+0000 7f44d3400700 1 -- 172.21.15.3:0/2106641660 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f44dc15b360 con 0x7f44dc071810 2024-09-13T16:53:15.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.795+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44dc15b360 con 0x7f44dc071810 2024-09-13T16:53:15.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.795+0000 7f44d3400700 1 -- 172.21.15.3:0/2106641660 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f44dc15b360 con 0x7f44dc071810 2024-09-13T16:53:15.802 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:15.802 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:15.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f44bc066730 msgr2=0x7f44bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:15.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f44bc066730 0x7f44bc068bf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f44c8007fd0 tx=0x7f44c8009040 comp rx=0 tx=0).stop 2024-09-13T16:53:15.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44dc071810 msgr2=0x7f44dc1574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:15.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44dc071810 0x7f44dc1574a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c580 tx=0x7f44cc00c940 comp rx=0 tx=0).stop 2024-09-13T16:53:15.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 shutdown_connections 2024-09-13T16:53:15.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f44bc066730 0x7f44bc068bf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:15.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44dc15cb40 0x7f44dc15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:15.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44dc0cf3b0 0x7f44dc15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:15.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 --2- 172.21.15.3:0/2106641660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44dc071810 0x7f44dc1574a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:15.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 >> 172.21.15.3:0/2106641660 conn(0x7f44dc06b400 msgr2=0x7f44dc158420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:15.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 shutdown_connections 2024-09-13T16:53:15.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:15.799+0000 7f44e0a00700 1 -- 172.21.15.3:0/2106641660 wait complete. 2024-09-13T16:53:15.807 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:15.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:16.026 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":27,"msgr_send_messages":25,"msgr_recv_bytes":15272,"msgr_send_bytes":26792,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.016665479,"msgr_running_send_time":0.002394879,"msgr_running_recv_time":0.007357952,"msgr_running_fast_dispatch_time":0.000834075,"msgr_send_messages_queue_lat":{"avgcount":24,"sum":0.009917775,"avgtime":0.000413240},"msgr_handle_ack_lat":{"avgcount":14,"sum":0.000004820,"avgtime":0.000000344},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":49,"msgr_send_messages":44,"msgr_recv_bytes":118134,"msgr_send_bytes":18539,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.013358846,"msgr_running_send_time":0.002676588,"msgr_running_recv_time":0.015897410,"msgr_running_fast_dispatch_time":0.000992481,"msgr_send_messages_queue_lat":{"avgcount":44,"sum":0.011034015,"avgtime":0.000250773},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":103888,"msgr_send_encrypted_bytes":6224},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":12,"msgr_recv_bytes":3670,"msgr_send_bytes":61556,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.005912886,"msgr_running_send_time":0.000899490,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":11,"sum":0.005451806,"avgtime":0.000495618},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":58880},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":50,"sum":0.007070057,"avgtime":0.000141401}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":3,"evex":0,"evtrm":0,"ev":5,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4198114,"rdpos":4195196,"jlat":{"avgcount":3,"sum":0.009454026,"avgtime":0.003151342},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":46168,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":951,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238382,"buffer_anon_items":72,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5784,"osdmap_items":93,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51904,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":47,"op_send_bytes":3197,"op_resend":0,"op_reply":47,"op_latency":{"avgcount":47,"sum":0.107998110,"avgtime":0.002297832},"op_inflight":0,"oplen_avg":{"avgcount":47,"sum":93},"op":47,"op_r":36,"op_w":11,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":1,"omap_rd":48,"omap_del":0},"oft":{"omap_total_objs":0,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":80,"get_sum":127200,"get_or_fail_fail":0,"get_or_fail_success":80,"take":0,"take_sum":0,"put":80,"put_sum":127200,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":4084,"put":47,"put_sum":4084,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":47,"put":47,"put_sum":47,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":3,"get_sum":2659,"get_or_fail_fail":0,"get_or_fail_success":3,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:16.028 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 14, "avgtime": 3.44e-07, "sum": 4.82e-06 }, "msgr_recv_bytes": 15272, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 27, "msgr_running_fast_dispatch_time": 0.000834075, "msgr_running_recv_time": 0.007357952, "msgr_running_send_time": 0.002394879, "msgr_running_total_time": 0.016665479, "msgr_send_bytes": 26792, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 25, "msgr_send_messages_queue_lat": { "avgcount": 24, "avgtime": 0.00041324, "sum": 0.009917775 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 118134, "msgr_recv_encrypted_bytes": 103888, "msgr_recv_messages": 49, "msgr_running_fast_dispatch_time": 0.000992481, "msgr_running_recv_time": 0.01589741, "msgr_running_send_time": 0.002676588, "msgr_running_total_time": 0.013358846, "msgr_send_bytes": 18539, "msgr_send_encrypted_bytes": 6224, "msgr_send_messages": 44, "msgr_send_messages_queue_lat": { "avgcount": 44, "avgtime": 0.000250773, "sum": 0.011034015 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.00089949, "msgr_running_total_time": 0.005912886, "msgr_send_bytes": 61556, "msgr_send_encrypted_bytes": 58880, "msgr_send_messages": 12, "msgr_send_messages_queue_lat": { "avgcount": 11, "avgtime": 0.000495618, "sum": 0.005451806 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 50, "avgtime": 0.000141401, "sum": 0.007070057 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 5, "evadd": 3, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.003151342, "sum": 0.009454026 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4198114 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 46168 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 951, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238382, "buffer_anon_items": 72, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51904, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5784, "osdmap_items": 93, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 0, "omap_rd": 48, "omap_wr": 1, "op": 47, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 47, "avgtime": 0.002297832, "sum": 0.10799811 }, "op_pg": 0, "op_r": 36, "op_reply": 47, "op_resend": 0, "op_rmw": 0, "op_send": 47, "op_send_bytes": 3197, "op_w": 11, "oplen_avg": { "avgcount": 47, "sum": 93 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 0, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 80, "get_or_fail_fail": 0, "get_or_fail_success": 80, "get_started": 0, "get_sum": 127200, "max": 104857600, "put": 80, "put_sum": 127200, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 47, "put_sum": 4084, "take": 47, "take_sum": 4084, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 47, "put_sum": 47, "take": 47, "take_sum": 47, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 3, "get_or_fail_fail": 0, "get_or_fail_success": 3, "get_started": 0, "get_sum": 2659, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:16.028 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T16:53:21.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:21.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.199+0000 7ff519a00700 1 Processor -- start 2024-09-13T16:53:21.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.199+0000 7ff519a00700 1 -- start start 2024-09-13T16:53:21.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff519a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff51406c570 0x7ff5141656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff519a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5140ce5d0 con 0x7ff514070dc0 2024-09-13T16:53:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff519a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51406ddc0 con 0x7ff51406ba80 2024-09-13T16:53:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff519a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51415af00 con 0x7ff51406c570 2024-09-13T16:53:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff513400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff514070dc0 0x7ff51406be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56124/0 (socket says 172.21.15.3:56124) 2024-09-13T16:53:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff513400700 1 -- 172.21.15.3:0/1615490826 learned_addr learned my addr 172.21.15.3:0/1615490826 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff519000700 1 --2- 172.21.15.3:0/1615490826 >> v2:172.21.15.157:3300/0 conn(0x7ff51406c570 0x7ff5141656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff512a00700 1 -- 172.21.15.3:0/1615490826 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338609559 0 0) 0x7ff5140ce5d0 con 0x7ff514070dc0 2024-09-13T16:53:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff512a00700 1 -- 172.21.15.3:0/1615490826 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4f80035d0 con 0x7ff514070dc0 2024-09-13T16:53:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff519000700 1 -- 172.21.15.3:0/1615490826 >> v1:172.21.15.157:6789/0 conn(0x7ff51406ba80 legacy=0x7ff51406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff519000700 1 -- 172.21.15.3:0/1615490826 >> v1:172.21.15.3:6789/0 conn(0x7ff514070dc0 legacy=0x7ff51406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff512a00700 1 -- 172.21.15.3:0/1615490826 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787401107 0 0) 0x7ff51406ddc0 con 0x7ff51406ba80 2024-09-13T16:53:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.203+0000 7ff519000700 1 -- 172.21.15.3:0/1615490826 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5140cd9c0 con 0x7ff51406c570 2024-09-13T16:53:21.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff519000700 1 --2- 172.21.15.3:0/1615490826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 0x7ff5141656d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009850 tx=0x7ff50c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb27c6ae742e8db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:21.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff512a00700 1 -- 172.21.15.3:0/1615490826 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c011830 con 0x7ff51406c570 2024-09-13T16:53:21.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff512a00700 1 -- 172.21.15.3:0/1615490826 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c011990 con 0x7ff51406c570 2024-09-13T16:53:21.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff512a00700 1 -- 172.21.15.3:0/1615490826 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c011d40 con 0x7ff51406c570 2024-09-13T16:53:21.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff519a00700 1 -- 172.21.15.3:0/1615490826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 msgr2=0x7ff5141656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff519a00700 1 --2- 172.21.15.3:0/1615490826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 0x7ff5141656d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009850 tx=0x7ff50c009b60 comp rx=0 tx=0).stop 2024-09-13T16:53:21.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff519a00700 1 -- 172.21.15.3:0/1615490826 shutdown_connections 2024-09-13T16:53:21.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff519a00700 1 --2- 172.21.15.3:0/1615490826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 0x7ff5141656d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff519a00700 1 -- 172.21.15.3:0/1615490826 >> 172.21.15.3:0/1615490826 conn(0x7ff51406a2f0 msgr2=0x7ff51406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:21.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff519a00700 1 -- 172.21.15.3:0/1615490826 shutdown_connections 2024-09-13T16:53:21.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.207+0000 7ff519a00700 1 -- 172.21.15.3:0/1615490826 wait complete. 2024-09-13T16:53:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519a00700 1 Processor -- start 2024-09-13T16:53:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519a00700 1 -- start start 2024-09-13T16:53:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51406ba80 0x7ff5141f9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 0x7ff514148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff514070dc0 0x7ff51406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514156c70 con 0x7ff514070dc0 2024-09-13T16:53:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514153db0 con 0x7ff51406c570 2024-09-13T16:53:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff514155ff0 con 0x7ff51406ba80 2024-09-13T16:53:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff514070dc0 0x7ff51406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff514070dc0 0x7ff51406cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55792/0 (socket says 172.21.15.3:55792) 2024-09-13T16:53:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519000700 1 -- 172.21.15.3:0/3278771903 learned_addr learned my addr 172.21.15.3:0/3278771903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff513400700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 0x7ff514148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff513e00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51406ba80 0x7ff5141f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519000700 1 -- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51406ba80 msgr2=0x7ff5141f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519000700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51406ba80 0x7ff5141f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519000700 1 -- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 msgr2=0x7ff514148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519000700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 0x7ff514148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff519000700 1 -- 172.21.15.3:0/3278771903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0094b0 con 0x7ff514070dc0 2024-09-13T16:53:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.211+0000 7ff513400700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 0x7ff514148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.215+0000 7ff519000700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff514070dc0 0x7ff51406cf90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff514072390 tx=0x7ff50c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.215+0000 7ff50be00700 1 -- 172.21.15.3:0/3278771903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c01a8d0 con 0x7ff514070dc0 2024-09-13T16:53:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.215+0000 7ff50be00700 1 -- 172.21.15.3:0/3278771903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c01f070 con 0x7ff514070dc0 2024-09-13T16:53:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.215+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5141562d0 con 0x7ff514070dc0 2024-09-13T16:53:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.215+0000 7ff50be00700 1 -- 172.21.15.3:0/3278771903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c01ac90 con 0x7ff514070dc0 2024-09-13T16:53:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.215+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff514156890 con 0x7ff514070dc0 2024-09-13T16:53:21.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.215+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff514156db0 con 0x7ff514070dc0 2024-09-13T16:53:21.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.215+0000 7ff50be00700 1 -- 172.21.15.3:0/3278771903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff50c01b430 con 0x7ff514070dc0 2024-09-13T16:53:21.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.219+0000 7ff50be00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4f4066730 0x7ff4f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.219+0000 7ff50be00700 1 -- 172.21.15.3:0/3278771903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff50c085470 con 0x7ff514070dc0 2024-09-13T16:53:21.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.219+0000 7ff513e00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4f4066730 0x7ff4f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.219+0000 7ff513e00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4f4066730 0x7ff4f4068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff5040012e0 tx=0x7ff504009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:21.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.223+0000 7ff50be00700 1 -- 172.21.15.3:0/3278771903 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff514156db0 con 0x7ff514070dc0 2024-09-13T16:53:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.459+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff514156db0 con 0x7ff514070dc0 2024-09-13T16:53:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.459+0000 7ff50be00700 1 -- 172.21.15.3:0/3278771903 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7ff514156db0 con 0x7ff514070dc0 2024-09-13T16:53:21.467 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:21.467 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:21.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4f4066730 msgr2=0x7ff4f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4f4066730 0x7ff4f4068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff5040012e0 tx=0x7ff504009040 comp rx=0 tx=0).stop 2024-09-13T16:53:21.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff514070dc0 msgr2=0x7ff51406cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff514070dc0 0x7ff51406cf90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff514072390 tx=0x7ff50c004770 comp rx=0 tx=0).stop 2024-09-13T16:53:21.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 shutdown_connections 2024-09-13T16:53:21.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4f4066730 0x7ff4f4068bf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff514070dc0 0x7ff51406cf90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51406c570 0x7ff514148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 --2- 172.21.15.3:0/3278771903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51406ba80 0x7ff5141f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 >> 172.21.15.3:0/3278771903 conn(0x7ff51406a2f0 msgr2=0x7ff5140aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:21.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 shutdown_connections 2024-09-13T16:53:21.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.463+0000 7ff519a00700 1 -- 172.21.15.3:0/3278771903 wait complete. 2024-09-13T16:53:21.473 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:21.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:21.684 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":37,"msgr_send_messages":30,"msgr_recv_bytes":19014,"msgr_send_bytes":29204,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.019331730,"msgr_running_send_time":0.003750651,"msgr_running_recv_time":0.007992396,"msgr_running_fast_dispatch_time":0.001460330,"msgr_send_messages_queue_lat":{"avgcount":29,"sum":0.010240717,"avgtime":0.000353128},"msgr_handle_ack_lat":{"avgcount":21,"sum":0.000006668,"avgtime":0.000000317},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":51,"msgr_send_messages":46,"msgr_recv_bytes":118652,"msgr_send_bytes":21167,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.013733819,"msgr_running_send_time":0.002878374,"msgr_running_recv_time":0.016022330,"msgr_running_fast_dispatch_time":0.001035424,"msgr_send_messages_queue_lat":{"avgcount":46,"sum":0.011233530,"avgtime":0.000244207},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":104160,"msgr_send_encrypted_bytes":6816},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":13,"msgr_recv_bytes":3670,"msgr_send_bytes":63332,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006020324,"msgr_running_send_time":0.000986689,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":12,"sum":0.005528071,"avgtime":0.000460672},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":60656},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.007557903,"avgtime":0.000139961}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199839,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.013203931,"avgtime":0.003300982},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":46552,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":956,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238864,"buffer_anon_items":74,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55175,"mds_co_items":229,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5458,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.131997652,"avgtime":0.002538416},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":92,"get_sum":130328,"get_or_fail_fail":0,"get_or_fail_success":92,"take":0,"take_sum":0,"put":92,"put_sum":130328,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6345,"put":52,"put_sum":6345,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4384,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4384,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:21.685 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 21, "avgtime": 3.17e-07, "sum": 6.668e-06 }, "msgr_recv_bytes": 19014, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 37, "msgr_running_fast_dispatch_time": 0.00146033, "msgr_running_recv_time": 0.007992396, "msgr_running_send_time": 0.003750651, "msgr_running_total_time": 0.01933173, "msgr_send_bytes": 29204, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 29, "avgtime": 0.000353128, "sum": 0.010240717 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 118652, "msgr_recv_encrypted_bytes": 104160, "msgr_recv_messages": 51, "msgr_running_fast_dispatch_time": 0.001035424, "msgr_running_recv_time": 0.01602233, "msgr_running_send_time": 0.002878374, "msgr_running_total_time": 0.013733819, "msgr_send_bytes": 21167, "msgr_send_encrypted_bytes": 6816, "msgr_send_messages": 46, "msgr_send_messages_queue_lat": { "avgcount": 46, "avgtime": 0.000244207, "sum": 0.01123353 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.000986689, "msgr_running_total_time": 0.006020324, "msgr_send_bytes": 63332, "msgr_send_encrypted_bytes": 60656, "msgr_send_messages": 13, "msgr_send_messages_queue_lat": { "avgcount": 12, "avgtime": 0.000460672, "sum": 0.005528071 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000139961, "sum": 0.007557903 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.003300982, "sum": 0.013203931 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4199839 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 46552 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 956, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238864, "buffer_anon_items": 74, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55175, "mds_co_items": 229, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.002538416, "sum": 0.131997652 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5458, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 92, "get_or_fail_fail": 0, "get_or_fail_success": 92, "get_started": 0, "get_sum": 130328, "max": 104857600, "put": 92, "put_sum": 130328, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6345, "take": 52, "take_sum": 6345, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4384, "max": 3758096384, "put": 4, "put_sum": 4384, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:21.685 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-13T16:53:21.685 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:21.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.851+0000 7fc488000700 1 Processor -- start 2024-09-13T16:53:21.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.851+0000 7fc488000700 1 -- start start 2024-09-13T16:53:21.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc488000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc480071e60 0x7fc48015dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc488000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4800ce470 0x7fc4800b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc488000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc480160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc488000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4800b2e60 con 0x7fc4800ce470 2024-09-13T16:53:21.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc488000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4800cd130 con 0x7fc480071e60 2024-09-13T16:53:21.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc488000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800cd3d0 con 0x7fc4800b29d0 2024-09-13T16:53:21.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4800ce470 0x7fc4800b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4800ce470 0x7fc4800b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55794/0 (socket says 172.21.15.3:55794) 2024-09-13T16:53:21.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 -- 172.21.15.3:0/3418263321 learned_addr learned my addr 172.21.15.3:0/3418263321 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:21.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc485600700 1 --2- 172.21.15.3:0/3418263321 >> v2:172.21.15.157:3300/0 conn(0x7fc480071e60 0x7fc48015dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc486000700 1 --2- 172.21.15.3:0/3418263321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc480160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 -- 172.21.15.3:0/3418263321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 msgr2=0x7fc480160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:21.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 --2- 172.21.15.3:0/3418263321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc480160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 -- 172.21.15.3:0/3418263321 >> v2:172.21.15.157:3300/0 conn(0x7fc480071e60 msgr2=0x7fc48015dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:21.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 --2- 172.21.15.3:0/3418263321 >> v2:172.21.15.157:3300/0 conn(0x7fc480071e60 0x7fc48015dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 -- 172.21.15.3:0/3418263321 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800cd6b0 con 0x7fc4800ce470 2024-09-13T16:53:21.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc484c00700 1 --2- 172.21.15.3:0/3418263321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc4800b2490 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc4800cc7c0 tx=0x7fc46c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832ef54a731a177e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:21.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.855+0000 7fc47fe00700 1 -- 172.21.15.3:0/3418263321 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c00b8a0 con 0x7fc4800ce470 2024-09-13T16:53:21.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc47fe00700 1 -- 172.21.15.3:0/3418263321 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c00ba00 con 0x7fc4800ce470 2024-09-13T16:53:21.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc47fe00700 1 -- 172.21.15.3:0/3418263321 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c00bd90 con 0x7fc4800ce470 2024-09-13T16:53:21.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 -- 172.21.15.3:0/3418263321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 msgr2=0x7fc4800b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 --2- 172.21.15.3:0/3418263321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc4800b2490 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc4800cc7c0 tx=0x7fc46c009bf0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 -- 172.21.15.3:0/3418263321 shutdown_connections 2024-09-13T16:53:21.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 --2- 172.21.15.3:0/3418263321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc4800b2490 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 --2- 172.21.15.3:0/3418263321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc480160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 --2- 172.21.15.3:0/3418263321 >> v2:172.21.15.157:3300/0 conn(0x7fc480071e60 0x7fc48015dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 -- 172.21.15.3:0/3418263321 >> 172.21.15.3:0/3418263321 conn(0x7fc4800b07a0 msgr2=0x7fc4800b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:21.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 -- 172.21.15.3:0/3418263321 shutdown_connections 2024-09-13T16:53:21.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.859+0000 7fc488000700 1 -- 172.21.15.3:0/3418263321 wait complete. 2024-09-13T16:53:21.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc488000700 1 Processor -- start 2024-09-13T16:53:21.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc488000700 1 -- start start 2024-09-13T16:53:21.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc488000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc480071e60 0x7fc48015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc488000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc4801e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc488000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc48014fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc488000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4801506c0 con 0x7fc4800ce470 2024-09-13T16:53:21.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc488000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4801e6cc0 con 0x7fc480071e60 2024-09-13T16:53:21.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc488000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015b320 con 0x7fc4800b29d0 2024-09-13T16:53:21.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc48014fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc48014fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55810/0 (socket says 172.21.15.3:55810) 2024-09-13T16:53:21.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 -- 172.21.15.3:0/4044377779 learned_addr learned my addr 172.21.15.3:0/4044377779 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:21.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc485600700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc480071e60 0x7fc48015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc484c00700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc4801e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 -- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 msgr2=0x7fc4801e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc4801e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 -- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc480071e60 msgr2=0x7fc48015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc480071e60 0x7fc48015ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 -- 172.21.15.3:0/4044377779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c009570 con 0x7fc4800ce470 2024-09-13T16:53:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc484c00700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc4801e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.863+0000 7fc486000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc48014fe80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc478007e60 tx=0x7fc47800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.867+0000 7fc47d600700 1 -- 172.21.15.3:0/4044377779 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47800ede0 con 0x7fc4800ce470 2024-09-13T16:53:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.867+0000 7fc47d600700 1 -- 172.21.15.3:0/4044377779 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc478014420 con 0x7fc4800ce470 2024-09-13T16:53:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.867+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48015b600 con 0x7fc4800ce470 2024-09-13T16:53:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.867+0000 7fc47d600700 1 -- 172.21.15.3:0/4044377779 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780147d0 con 0x7fc4800ce470 2024-09-13T16:53:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.867+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48015bbc0 con 0x7fc4800ce470 2024-09-13T16:53:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.867+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48006dee0 con 0x7fc4800ce470 2024-09-13T16:53:21.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.871+0000 7fc47d600700 1 -- 172.21.15.3:0/4044377779 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc478014930 con 0x7fc4800ce470 2024-09-13T16:53:21.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.871+0000 7fc47d600700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc460066730 0x7fc460068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:21.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.871+0000 7fc47d600700 1 -- 172.21.15.3:0/4044377779 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fc478084a40 con 0x7fc4800ce470 2024-09-13T16:53:21.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.871+0000 7fc485600700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc460066730 0x7fc460068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:21.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.871+0000 7fc485600700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc460066730 0x7fc460068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc48015d1f0 tx=0x7fc474008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:21.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:21.879+0000 7fc47d600700 1 -- 172.21.15.3:0/4044377779 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc478086e60 con 0x7fc4800ce470 2024-09-13T16:53:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.107+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc48006d960 con 0x7fc4800ce470 2024-09-13T16:53:22.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.107+0000 7fc47d600700 1 -- 172.21.15.3:0/4044377779 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fc478051c10 con 0x7fc4800ce470 2024-09-13T16:53:22.114 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:22.114 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc460066730 msgr2=0x7fc460068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc460066730 0x7fc460068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc48015d1f0 tx=0x7fc474008040 comp rx=0 tx=0).stop 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 msgr2=0x7fc48014fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc48014fe80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc478007e60 tx=0x7fc47800e3c0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 shutdown_connections 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc460066730 0x7fc460068bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4800ce470 0x7fc48014fe80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b29d0 0x7fc4801e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 --2- 172.21.15.3:0/4044377779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc480071e60 0x7fc48015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 >> 172.21.15.3:0/4044377779 conn(0x7fc4800b07a0 msgr2=0x7fc48006cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 shutdown_connections 2024-09-13T16:53:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.111+0000 7fc488000700 1 -- 172.21.15.3:0/4044377779 wait complete. 2024-09-13T16:53:22.118 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:22.138 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:22.321 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":38,"msgr_send_messages":30,"msgr_recv_bytes":19432,"msgr_send_bytes":29204,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.019525693,"msgr_running_send_time":0.003838072,"msgr_running_recv_time":0.008037026,"msgr_running_fast_dispatch_time":0.001518608,"msgr_send_messages_queue_lat":{"avgcount":29,"sum":0.010240717,"avgtime":0.000353128},"msgr_handle_ack_lat":{"avgcount":22,"sum":0.000006819,"avgtime":0.000000309},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":51,"msgr_send_messages":46,"msgr_recv_bytes":118652,"msgr_send_bytes":21167,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.013870780,"msgr_running_send_time":0.002943193,"msgr_running_recv_time":0.016055570,"msgr_running_fast_dispatch_time":0.001035424,"msgr_send_messages_queue_lat":{"avgcount":46,"sum":0.011233530,"avgtime":0.000244207},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":104160,"msgr_send_encrypted_bytes":6816},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":13,"msgr_recv_bytes":3670,"msgr_send_bytes":63332,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006020324,"msgr_running_send_time":0.000986689,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":12,"sum":0.005528071,"avgtime":0.000460672},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":60656},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.007557903,"avgtime":0.000139961}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199839,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.013203931,"avgtime":0.003300982},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":46552,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":956,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238864,"buffer_anon_items":74,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55175,"mds_co_items":229,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5458,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.131997652,"avgtime":0.002538416},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":93,"get_sum":130656,"get_or_fail_fail":0,"get_or_fail_success":93,"take":0,"take_sum":0,"put":93,"put_sum":130656,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6345,"put":52,"put_sum":6345,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4384,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4384,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:22.323 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 22, "avgtime": 3.09e-07, "sum": 6.819e-06 }, "msgr_recv_bytes": 19432, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 38, "msgr_running_fast_dispatch_time": 0.001518608, "msgr_running_recv_time": 0.008037026, "msgr_running_send_time": 0.003838072, "msgr_running_total_time": 0.019525693, "msgr_send_bytes": 29204, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 29, "avgtime": 0.000353128, "sum": 0.010240717 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 118652, "msgr_recv_encrypted_bytes": 104160, "msgr_recv_messages": 51, "msgr_running_fast_dispatch_time": 0.001035424, "msgr_running_recv_time": 0.01605557, "msgr_running_send_time": 0.002943193, "msgr_running_total_time": 0.01387078, "msgr_send_bytes": 21167, "msgr_send_encrypted_bytes": 6816, "msgr_send_messages": 46, "msgr_send_messages_queue_lat": { "avgcount": 46, "avgtime": 0.000244207, "sum": 0.01123353 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.000986689, "msgr_running_total_time": 0.006020324, "msgr_send_bytes": 63332, "msgr_send_encrypted_bytes": 60656, "msgr_send_messages": 13, "msgr_send_messages_queue_lat": { "avgcount": 12, "avgtime": 0.000460672, "sum": 0.005528071 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000139961, "sum": 0.007557903 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.003300982, "sum": 0.013203931 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4199839 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 46552 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 956, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238864, "buffer_anon_items": 74, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55175, "mds_co_items": 229, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.002538416, "sum": 0.131997652 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5458, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 93, "get_or_fail_fail": 0, "get_or_fail_success": 93, "get_started": 0, "get_sum": 130656, "max": 104857600, "put": 93, "put_sum": 130656, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6345, "take": 52, "take_sum": 6345, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4384, "max": 3758096384, "put": 4, "put_sum": 4384, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:22.323 INFO:tasks.cephfs.test_openfiletable:omap_total_updates_1:2 2024-09-13T16:53:22.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:22.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb989e00700 1 Processor -- start 2024-09-13T16:53:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb989e00700 1 -- start start 2024-09-13T16:53:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb989e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9840ad3d0 0x7fb98406aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb989e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9840ca290 0x7fb9840ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb989e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98415ae80 con 0x7fb98415aaa0 2024-09-13T16:53:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb989e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98406af40 con 0x7fb9840ad3d0 2024-09-13T16:53:22.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb989e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98406b200 con 0x7fb9840ca290 2024-09-13T16:53:22.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb983e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9840ad3d0 0x7fb98406aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:22.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb983e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9840ad3d0 0x7fb98406aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55820/0 (socket says 172.21.15.3:55820) 2024-09-13T16:53:22.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb983e00700 1 -- 172.21.15.3:0/73194188 learned_addr learned my addr 172.21.15.3:0/73194188 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:22.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.551+0000 7fb989400700 1 --2- 172.21.15.3:0/73194188 >> v2:172.21.15.157:3300/0 conn(0x7fb9840ca290 0x7fb9840ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:22.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb983e00700 1 -- 172.21.15.3:0/73194188 >> v2:172.21.15.157:3300/0 conn(0x7fb9840ca290 msgr2=0x7fb9840ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:22.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb983e00700 1 --2- 172.21.15.3:0/73194188 >> v2:172.21.15.157:3300/0 conn(0x7fb9840ca290 0x7fb9840ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb983e00700 1 -- 172.21.15.3:0/73194188 >> v1:172.21.15.3:6789/0 conn(0x7fb98415aaa0 legacy=0x7fb98406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:22.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb983e00700 1 -- 172.21.15.3:0/73194188 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98406b4e0 con 0x7fb9840ad3d0 2024-09-13T16:53:22.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb983e00700 1 --2- 172.21.15.3:0/73194188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb98406aa00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb98406e340 tx=0x7fb970009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abe3d4cf45df5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:22.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb983400700 1 -- 172.21.15.3:0/73194188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97000b8a0 con 0x7fb9840ad3d0 2024-09-13T16:53:22.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb983400700 1 -- 172.21.15.3:0/73194188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97000ba00 con 0x7fb9840ad3d0 2024-09-13T16:53:22.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb983400700 1 -- 172.21.15.3:0/73194188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97000bd90 con 0x7fb9840ad3d0 2024-09-13T16:53:22.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb989e00700 1 -- 172.21.15.3:0/73194188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 msgr2=0x7fb98406aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:22.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb989e00700 1 --2- 172.21.15.3:0/73194188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb98406aa00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb98406e340 tx=0x7fb970009bf0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb989e00700 1 -- 172.21.15.3:0/73194188 shutdown_connections 2024-09-13T16:53:22.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb989e00700 1 --2- 172.21.15.3:0/73194188 >> v2:172.21.15.157:3300/0 conn(0x7fb9840ca290 0x7fb9840ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb989e00700 1 --2- 172.21.15.3:0/73194188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb98406aa00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb989e00700 1 -- 172.21.15.3:0/73194188 >> 172.21.15.3:0/73194188 conn(0x7fb9840cf010 msgr2=0x7fb9840cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:22.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb989e00700 1 -- 172.21.15.3:0/73194188 shutdown_connections 2024-09-13T16:53:22.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.555+0000 7fb989e00700 1 -- 172.21.15.3:0/73194188 wait complete. 2024-09-13T16:53:22.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.559+0000 7fb989e00700 1 Processor -- start 2024-09-13T16:53:22.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.559+0000 7fb989e00700 1 -- start start 2024-09-13T16:53:22.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb9841f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:22.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9840ca290 0x7fb98409bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:22.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb98415aaa0 0x7fb9840cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:22.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841f3c50 con 0x7fb9840ad3d0 2024-09-13T16:53:22.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840b30d0 con 0x7fb9840ca290 2024-09-13T16:53:22.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840b24c0 con 0x7fb98415aaa0 2024-09-13T16:53:22.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb9841f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:22.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb9841f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55828/0 (socket says 172.21.15.3:55828) 2024-09-13T16:53:22.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 -- 172.21.15.3:0/2541921953 learned_addr learned my addr 172.21.15.3:0/2541921953 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989400700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb98415aaa0 0x7fb9840cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb983e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9840ca290 0x7fb98409bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 -- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb98415aaa0 msgr2=0x7fb9840cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb98415aaa0 0x7fb9840cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 -- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9840ca290 msgr2=0x7fb98409bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9840ca290 0x7fb98409bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 -- 172.21.15.3:0/2541921953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970009570 con 0x7fb9840ad3d0 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb983e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9840ca290 0x7fb98409bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb988a00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb9841f3710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c410 tx=0x7fb97800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb980c00700 1 -- 172.21.15.3:0/2541921953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9780052d0 con 0x7fb9840ad3d0 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb980c00700 1 -- 172.21.15.3:0/2541921953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb978005430 con 0x7fb9840ad3d0 2024-09-13T16:53:22.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb980c00700 1 -- 172.21.15.3:0/2541921953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9780055a0 con 0x7fb9840ad3d0 2024-09-13T16:53:22.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9840b27a0 con 0x7fb9840ad3d0 2024-09-13T16:53:22.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.563+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9840b2d60 con 0x7fb9840ad3d0 2024-09-13T16:53:22.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.567+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98406aac0 con 0x7fb9840ad3d0 2024-09-13T16:53:22.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.567+0000 7fb980c00700 1 -- 172.21.15.3:0/2541921953 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb9780036a0 con 0x7fb9840ad3d0 2024-09-13T16:53:22.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.567+0000 7fb980c00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb964066760 0x7fb964068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:22.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.567+0000 7fb980c00700 1 -- 172.21.15.3:0/2541921953 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fb978083690 con 0x7fb9840ad3d0 2024-09-13T16:53:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.571+0000 7fb983e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb964066760 0x7fb964068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:22.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.571+0000 7fb983e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb964066760 0x7fb964068c20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb98406b890 tx=0x7fb970008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:22.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.575+0000 7fb980c00700 1 -- 172.21.15.3:0/2541921953 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb978050730 con 0x7fb9840ad3d0 2024-09-13T16:53:22.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.811+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb984072650 con 0x7fb9840ad3d0 2024-09-13T16:53:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.811+0000 7fb980c00700 1 -- 172.21.15.3:0/2541921953 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fb978053d50 con 0x7fb9840ad3d0 2024-09-13T16:53:22.818 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:22.818 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:22.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb964066760 msgr2=0x7fb964068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:22.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb964066760 0x7fb964068c20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb98406b890 tx=0x7fb970008030 comp rx=0 tx=0).stop 2024-09-13T16:53:22.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 msgr2=0x7fb9841f3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:22.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb9841f3710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c410 tx=0x7fb97800c7d0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 shutdown_connections 2024-09-13T16:53:22.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb964066760 0x7fb964068c20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb98415aaa0 0x7fb9840cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9840ca290 0x7fb98409bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 --2- 172.21.15.3:0/2541921953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9840ad3d0 0x7fb9841f3710 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 >> 172.21.15.3:0/2541921953 conn(0x7fb9840cf010 msgr2=0x7fb9840aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 shutdown_connections 2024-09-13T16:53:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:22.815+0000 7fb989e00700 1 -- 172.21.15.3:0/2541921953 wait complete. 2024-09-13T16:53:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:22.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:23.029 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":39,"msgr_send_messages":30,"msgr_recv_bytes":19850,"msgr_send_bytes":29204,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.019815311,"msgr_running_send_time":0.003981095,"msgr_running_recv_time":0.008096388,"msgr_running_fast_dispatch_time":0.001601777,"msgr_send_messages_queue_lat":{"avgcount":29,"sum":0.010240717,"avgtime":0.000353128},"msgr_handle_ack_lat":{"avgcount":23,"sum":0.000007059,"avgtime":0.000000306},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":51,"msgr_send_messages":46,"msgr_recv_bytes":118652,"msgr_send_bytes":21167,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.013870780,"msgr_running_send_time":0.002943193,"msgr_running_recv_time":0.016055570,"msgr_running_fast_dispatch_time":0.001035424,"msgr_send_messages_queue_lat":{"avgcount":46,"sum":0.011233530,"avgtime":0.000244207},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":104160,"msgr_send_encrypted_bytes":6816},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":13,"msgr_recv_bytes":3670,"msgr_send_bytes":63332,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006020324,"msgr_running_send_time":0.000986689,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":12,"sum":0.005528071,"avgtime":0.000460672},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":60656},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.007557903,"avgtime":0.000139961}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199839,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.013203931,"avgtime":0.003300982},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":46552,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":956,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238864,"buffer_anon_items":74,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55175,"mds_co_items":229,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5458,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.131997652,"avgtime":0.002538416},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":94,"get_sum":130984,"get_or_fail_fail":0,"get_or_fail_success":94,"take":0,"take_sum":0,"put":94,"put_sum":130984,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6345,"put":52,"put_sum":6345,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4384,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4384,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:23.030 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 23, "avgtime": 3.06e-07, "sum": 7.059e-06 }, "msgr_recv_bytes": 19850, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 39, "msgr_running_fast_dispatch_time": 0.001601777, "msgr_running_recv_time": 0.008096388, "msgr_running_send_time": 0.003981095, "msgr_running_total_time": 0.019815311, "msgr_send_bytes": 29204, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 29, "avgtime": 0.000353128, "sum": 0.010240717 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 118652, "msgr_recv_encrypted_bytes": 104160, "msgr_recv_messages": 51, "msgr_running_fast_dispatch_time": 0.001035424, "msgr_running_recv_time": 0.01605557, "msgr_running_send_time": 0.002943193, "msgr_running_total_time": 0.01387078, "msgr_send_bytes": 21167, "msgr_send_encrypted_bytes": 6816, "msgr_send_messages": 46, "msgr_send_messages_queue_lat": { "avgcount": 46, "avgtime": 0.000244207, "sum": 0.01123353 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.000986689, "msgr_running_total_time": 0.006020324, "msgr_send_bytes": 63332, "msgr_send_encrypted_bytes": 60656, "msgr_send_messages": 13, "msgr_send_messages_queue_lat": { "avgcount": 12, "avgtime": 0.000460672, "sum": 0.005528071 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000139961, "sum": 0.007557903 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.003300982, "sum": 0.013203931 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4199839 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 46552 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 956, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238864, "buffer_anon_items": 74, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55175, "mds_co_items": 229, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.002538416, "sum": 0.131997652 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5458, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 94, "get_or_fail_fail": 0, "get_or_fail_success": 94, "get_started": 0, "get_sum": 130984, "max": 104857600, "put": 94, "put_sum": 130984, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6345, "take": 52, "take_sum": 6345, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4384, "max": 3758096384, "put": 4, "put_sum": 4384, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:23.030 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T16:53:28.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:28.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.203+0000 7fcb05800700 1 Processor -- start 2024-09-13T16:53:28.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.203+0000 7fcb05800700 1 -- start start 2024-09-13T16:53:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcb05800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb00070dc0 0x7fcb0006be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcb05800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb000ce5d0 con 0x7fcb0006c3a0 2024-09-13T16:53:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcb05800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0006ddc0 con 0x7fcb0006ba80 2024-09-13T16:53:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcb05800700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0015af00 con 0x7fcb00070dc0 2024-09-13T16:53:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcb04e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcb0006c3a0 0x7fcb001646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41922/0 (socket says 172.21.15.3:41922) 2024-09-13T16:53:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcb04e00700 1 -- 172.21.15.3:0/2483753319 learned_addr learned my addr 172.21.15.3:0/2483753319 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcaff400700 1 --2- 172.21.15.3:0/2483753319 >> v2:172.21.15.157:3300/0 conn(0x7fcb00070dc0 0x7fcb0006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcafea00700 1 -- 172.21.15.3:0/2483753319 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004915878 0 0) 0x7fcb000ce5d0 con 0x7fcb0006c3a0 2024-09-13T16:53:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcafea00700 1 -- 172.21.15.3:0/2483753319 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae40035d0 con 0x7fcb0006c3a0 2024-09-13T16:53:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcaff400700 1 -- 172.21.15.3:0/2483753319 >> v1:172.21.15.157:6789/0 conn(0x7fcb0006ba80 legacy=0x7fcb0006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcaff400700 1 -- 172.21.15.3:0/2483753319 >> v1:172.21.15.3:6789/0 conn(0x7fcb0006c3a0 legacy=0x7fcb001646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.207+0000 7fcaff400700 1 -- 172.21.15.3:0/2483753319 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb000cd9c0 con 0x7fcb00070dc0 2024-09-13T16:53:28.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcaff400700 1 --2- 172.21.15.3:0/2483753319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 0x7fcb0006be60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009850 tx=0x7fcaec009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62090822f6009c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcafea00700 1 -- 172.21.15.3:0/2483753319 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011830 con 0x7fcb00070dc0 2024-09-13T16:53:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcafea00700 1 -- 172.21.15.3:0/2483753319 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec011990 con 0x7fcb00070dc0 2024-09-13T16:53:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcafea00700 1 -- 172.21.15.3:0/2483753319 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011d40 con 0x7fcb00070dc0 2024-09-13T16:53:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcb05800700 1 -- 172.21.15.3:0/2483753319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 msgr2=0x7fcb0006be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcb05800700 1 --2- 172.21.15.3:0/2483753319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 0x7fcb0006be60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009850 tx=0x7fcaec009b60 comp rx=0 tx=0).stop 2024-09-13T16:53:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcb05800700 1 -- 172.21.15.3:0/2483753319 shutdown_connections 2024-09-13T16:53:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcb05800700 1 --2- 172.21.15.3:0/2483753319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 0x7fcb0006be60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcb05800700 1 -- 172.21.15.3:0/2483753319 >> 172.21.15.3:0/2483753319 conn(0x7fcb0006a2f0 msgr2=0x7fcb0006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcb05800700 1 -- 172.21.15.3:0/2483753319 shutdown_connections 2024-09-13T16:53:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.211+0000 7fcb05800700 1 -- 172.21.15.3:0/2483753319 wait complete. 2024-09-13T16:53:28.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb05800700 1 Processor -- start 2024-09-13T16:53:28.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb05800700 1 -- start start 2024-09-13T16:53:28.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb05800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb0006ba80 0x7fcb001e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:28.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb05800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0006c3a0 0x7fcb00142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:28.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb05800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 0x7fcb000ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:28.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb05800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00143d60 con 0x7fcb0006c3a0 2024-09-13T16:53:28.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb05800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb001437a0 con 0x7fcb00070dc0 2024-09-13T16:53:28.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb05800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00143a80 con 0x7fcb0006ba80 2024-09-13T16:53:28.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0006c3a0 0x7fcb00142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:28.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0006c3a0 0x7fcb00142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37310/0 (socket says 172.21.15.3:37310) 2024-09-13T16:53:28.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 -- 172.21.15.3:0/679671406 learned_addr learned my addr 172.21.15.3:0/679671406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:28.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcb04e00700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 0x7fcb000ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:28.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaffe00700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb0006ba80 0x7fcb001e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:28.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 -- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb0006ba80 msgr2=0x7fcb001e8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:28.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb0006ba80 0x7fcb001e8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:28.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 -- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 msgr2=0x7fcb000ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:28.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 0x7fcb000ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:28.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 -- 172.21.15.3:0/679671406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec0094b0 con 0x7fcb0006c3a0 2024-09-13T16:53:28.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaffe00700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb0006ba80 0x7fcb001e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:28.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.215+0000 7fcaff400700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0006c3a0 0x7fcb00142df0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcaec014040 tx=0x7fcaec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:28.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.219+0000 7fcaf7e00700 1 -- 172.21.15.3:0/679671406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec004510 con 0x7fcb0006c3a0 2024-09-13T16:53:28.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.219+0000 7fcaf7e00700 1 -- 172.21.15.3:0/679671406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec01e070 con 0x7fcb0006c3a0 2024-09-13T16:53:28.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.219+0000 7fcaf7e00700 1 -- 172.21.15.3:0/679671406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec003900 con 0x7fcb0006c3a0 2024-09-13T16:53:28.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.219+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb00140e60 con 0x7fcb0006c3a0 2024-09-13T16:53:28.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.219+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb001e90a0 con 0x7fcb0006c3a0 2024-09-13T16:53:28.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.219+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00141380 con 0x7fcb0006c3a0 2024-09-13T16:53:28.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.219+0000 7fcaf7e00700 1 -- 172.21.15.3:0/679671406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcaec003a60 con 0x7fcb0006c3a0 2024-09-13T16:53:28.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.223+0000 7fcaf7e00700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcae0066730 0x7fcae0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:28.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.223+0000 7fcaf7e00700 1 -- 172.21.15.3:0/679671406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fcaec085210 con 0x7fcb0006c3a0 2024-09-13T16:53:28.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.223+0000 7fcaffe00700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcae0066730 0x7fcae0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:28.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.223+0000 7fcaffe00700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcae0066730 0x7fcae0068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00017a0 tx=0x7fcaf0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.231+0000 7fcaf7e00700 1 -- 172.21.15.3:0/679671406 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb00141380 con 0x7fcb0006c3a0 2024-09-13T16:53:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.459+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb00141380 con 0x7fcb0006c3a0 2024-09-13T16:53:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.459+0000 7fcaf7e00700 1 -- 172.21.15.3:0/679671406 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fcb00141380 con 0x7fcb0006c3a0 2024-09-13T16:53:28.468 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcae0066730 msgr2=0x7fcae0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcae0066730 0x7fcae0068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00017a0 tx=0x7fcaf0009040 comp rx=0 tx=0).stop 2024-09-13T16:53:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0006c3a0 msgr2=0x7fcb00142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0006c3a0 0x7fcb00142df0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcaec014040 tx=0x7fcaec004770 comp rx=0 tx=0).stop 2024-09-13T16:53:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 shutdown_connections 2024-09-13T16:53:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcae0066730 0x7fcae0068bf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00070dc0 0x7fcb000ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0006c3a0 0x7fcb00142df0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 --2- 172.21.15.3:0/679671406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb0006ba80 0x7fcb001e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 >> 172.21.15.3:0/679671406 conn(0x7fcb0006a2f0 msgr2=0x7fcb0006a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 shutdown_connections 2024-09-13T16:53:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:28.463+0000 7fcb05800700 1 -- 172.21.15.3:0/679671406 wait complete. 2024-09-13T16:53:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:28.492 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:28.692 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":45,"msgr_send_messages":31,"msgr_recv_bytes":22210,"msgr_send_bytes":29322,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.021661903,"msgr_running_send_time":0.004936216,"msgr_running_recv_time":0.008557265,"msgr_running_fast_dispatch_time":0.002032093,"msgr_send_messages_queue_lat":{"avgcount":30,"sum":0.010346982,"avgtime":0.000344899},"msgr_handle_ack_lat":{"avgcount":30,"sum":0.000009412,"avgtime":0.000000313},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":53,"msgr_send_messages":48,"msgr_recv_bytes":119196,"msgr_send_bytes":22351,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.014216409,"msgr_running_send_time":0.003119771,"msgr_running_recv_time":0.016260772,"msgr_running_fast_dispatch_time":0.001035424,"msgr_send_messages_queue_lat":{"avgcount":48,"sum":0.011449182,"avgtime":0.000238524},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":104704,"msgr_send_encrypted_bytes":8000},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":14,"msgr_recv_bytes":3670,"msgr_send_bytes":65108,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006132142,"msgr_running_send_time":0.001078330,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":13,"sum":0.005606849,"avgtime":0.000431296},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":62432},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.007557903,"avgtime":0.000139961}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199839,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.013203931,"avgtime":0.003300982},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":46936,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":961,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238864,"buffer_anon_items":74,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55215,"mds_co_items":234,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5458,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.131997652,"avgtime":0.002538416},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":102,"get_sum":133064,"get_or_fail_fail":0,"get_or_fail_success":102,"take":0,"take_sum":0,"put":102,"put_sum":133064,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6345,"put":52,"put_sum":6345,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4384,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4384,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:28.694 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 30, "avgtime": 3.13e-07, "sum": 9.412e-06 }, "msgr_recv_bytes": 22210, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 45, "msgr_running_fast_dispatch_time": 0.002032093, "msgr_running_recv_time": 0.008557265, "msgr_running_send_time": 0.004936216, "msgr_running_total_time": 0.021661903, "msgr_send_bytes": 29322, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 31, "msgr_send_messages_queue_lat": { "avgcount": 30, "avgtime": 0.000344899, "sum": 0.010346982 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 119196, "msgr_recv_encrypted_bytes": 104704, "msgr_recv_messages": 53, "msgr_running_fast_dispatch_time": 0.001035424, "msgr_running_recv_time": 0.016260772, "msgr_running_send_time": 0.003119771, "msgr_running_total_time": 0.014216409, "msgr_send_bytes": 22351, "msgr_send_encrypted_bytes": 8000, "msgr_send_messages": 48, "msgr_send_messages_queue_lat": { "avgcount": 48, "avgtime": 0.000238524, "sum": 0.011449182 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.00107833, "msgr_running_total_time": 0.006132142, "msgr_send_bytes": 65108, "msgr_send_encrypted_bytes": 62432, "msgr_send_messages": 14, "msgr_send_messages_queue_lat": { "avgcount": 13, "avgtime": 0.000431296, "sum": 0.005606849 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000139961, "sum": 0.007557903 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.003300982, "sum": 0.013203931 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4199839 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 46936 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 961, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238864, "buffer_anon_items": 74, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55215, "mds_co_items": 234, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.002538416, "sum": 0.131997652 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5458, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 102, "get_or_fail_fail": 0, "get_or_fail_success": 102, "get_started": 0, "get_sum": 133064, "max": 104857600, "put": 102, "put_sum": 133064, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6345, "take": 52, "take_sum": 6345, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4384, "max": 3758096384, "put": 4, "put_sum": 4384, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:28.694 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T16:53:28.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:53:28.882 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:53:33.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.875+0000 7ff6f7e00700 1 Processor -- start 2024-09-13T16:53:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.875+0000 7ff6f7e00700 1 -- start start 2024-09-13T16:53:33.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.875+0000 7ff6f7e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815a6d0 0x7ff6f815eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:33.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.875+0000 7ff6f7e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6f815f060 0x7ff6f806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:33.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.875+0000 7ff6f7e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f815b670 con 0x7ff6f80cc260 2024-09-13T16:53:33.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6f7e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f80cc810 con 0x7ff6f815f060 2024-09-13T16:53:33.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6f7e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f815b950 con 0x7ff6f815a6d0 2024-09-13T16:53:33.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6ee000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6f815f060 0x7ff6f806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:33.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6ee000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6f815f060 0x7ff6f806d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57660/0 (socket says 172.21.15.3:57660) 2024-09-13T16:53:33.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6ee000700 1 -- 172.21.15.3:0/3149107763 learned_addr learned my addr 172.21.15.3:0/3149107763 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:33.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6f6a00700 1 --2- 172.21.15.3:0/3149107763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815a6d0 0x7ff6f815eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:33.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6ee000700 1 -- 172.21.15.3:0/3149107763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815a6d0 msgr2=0x7ff6f815eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:33.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6ee000700 1 --2- 172.21.15.3:0/3149107763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815a6d0 0x7ff6f815eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:33.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6ee000700 1 -- 172.21.15.3:0/3149107763 >> v1:172.21.15.157:6789/0 conn(0x7ff6f80cc260 legacy=0x7ff6f815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:33.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6ee000700 1 -- 172.21.15.3:0/3149107763 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f815bc30 con 0x7ff6f815f060 2024-09-13T16:53:33.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6ee000700 1 --2- 172.21.15.3:0/3149107763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815f060 0x7ff6f806d060 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009e50 tx=0x7ff6e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969cd38249935142 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:33.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6f6000700 1 -- 172.21.15.3:0/3149107763 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000e070 con 0x7ff6f815f060 2024-09-13T16:53:33.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6f6000700 1 -- 172.21.15.3:0/3149107763 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e000b910 con 0x7ff6f815f060 2024-09-13T16:53:33.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.879+0000 7ff6f6000700 1 -- 172.21.15.3:0/3149107763 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000bcc0 con 0x7ff6f815f060 2024-09-13T16:53:33.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3149107763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815f060 msgr2=0x7ff6f806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:33.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3149107763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815f060 0x7ff6f806d060 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009e50 tx=0x7ff6e000b330 comp rx=0 tx=0).stop 2024-09-13T16:53:33.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3149107763 shutdown_connections 2024-09-13T16:53:33.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3149107763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815f060 0x7ff6f806d060 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:33.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3149107763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815a6d0 0x7ff6f815eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:33.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3149107763 >> 172.21.15.3:0/3149107763 conn(0x7ff6f806b400 msgr2=0x7ff6f806b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:33.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3149107763 shutdown_connections 2024-09-13T16:53:33.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3149107763 wait complete. 2024-09-13T16:53:33.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 Processor -- start 2024-09-13T16:53:33.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.883+0000 7ff6f7e00700 1 -- start start 2024-09-13T16:53:33.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f80cc260 0x7ff6f81f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:33.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815a6d0 0x7ff6f81e6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:33.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815f060 0x7ff6f8158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:33.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f81e4340 con 0x7ff6f815a6d0 2024-09-13T16:53:33.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f81f2ce0 con 0x7ff6f80cc260 2024-09-13T16:53:33.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f81e6e90 con 0x7ff6f815f060 2024-09-13T16:53:33.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815a6d0 0x7ff6f81e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:33.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815a6d0 0x7ff6f81e6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57664/0 (socket says 172.21.15.3:57664) 2024-09-13T16:53:33.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 -- 172.21.15.3:0/3172596696 learned_addr learned my addr 172.21.15.3:0/3172596696 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:33.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f6a00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f80cc260 0x7ff6f81f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:33.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7400700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815f060 0x7ff6f8158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:33.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 -- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815f060 msgr2=0x7ff6f8158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815f060 0x7ff6f8158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 -- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f80cc260 msgr2=0x7ff6f81f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f80cc260 0x7ff6f81f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 -- 172.21.15.3:0/3172596696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0009b30 con 0x7ff6f815a6d0 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ee000700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815a6d0 0x7ff6f81e6600 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000b370 tx=0x7ff6e0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ef400700 1 -- 172.21.15.3:0/3172596696 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00048b0 con 0x7ff6f815a6d0 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ef400700 1 -- 172.21.15.3:0/3172596696 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e00092e0 con 0x7ff6f815a6d0 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6ef400700 1 -- 172.21.15.3:0/3172596696 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e001c680 con 0x7ff6f815a6d0 2024-09-13T16:53:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f81e45e0 con 0x7ff6f815a6d0 2024-09-13T16:53:33.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.887+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f81e6b40 con 0x7ff6f815a6d0 2024-09-13T16:53:33.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.891+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f815eb20 con 0x7ff6f815a6d0 2024-09-13T16:53:33.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.891+0000 7ff6ef400700 1 -- 172.21.15.3:0/3172596696 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff6e001c7e0 con 0x7ff6f815a6d0 2024-09-13T16:53:33.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.891+0000 7ff6ef400700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff6d4066730 0x7ff6d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:33.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.891+0000 7ff6ef400700 1 -- 172.21.15.3:0/3172596696 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff6e0085cc0 con 0x7ff6f815a6d0 2024-09-13T16:53:33.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.895+0000 7ff6f6a00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff6d4066730 0x7ff6d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:33.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.895+0000 7ff6f6a00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff6d4066730 0x7ff6d4068bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff6f81e5e30 tx=0x7ff6e8009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:33.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:33.899+0000 7ff6ef400700 1 -- 172.21.15.3:0/3172596696 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff6e008c080 con 0x7ff6f815a6d0 2024-09-13T16:53:34.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.131+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6f8071e90 con 0x7ff6f815a6d0 2024-09-13T16:53:34.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.131+0000 7ff6ef400700 1 -- 172.21.15.3:0/3172596696 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7ff6e0052df0 con 0x7ff6f815a6d0 2024-09-13T16:53:34.140 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:34.140 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff6d4066730 msgr2=0x7ff6d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff6d4066730 0x7ff6d4068bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff6f81e5e30 tx=0x7ff6e8009240 comp rx=0 tx=0).stop 2024-09-13T16:53:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815a6d0 msgr2=0x7ff6f81e6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815a6d0 0x7ff6f81e6600 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000b370 tx=0x7ff6e0004af0 comp rx=0 tx=0).stop 2024-09-13T16:53:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 shutdown_connections 2024-09-13T16:53:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff6d4066730 0x7ff6d4068bf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815f060 0x7ff6f8158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6f815a6d0 0x7ff6f81e6600 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 --2- 172.21.15.3:0/3172596696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f80cc260 0x7ff6f81f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 >> 172.21.15.3:0/3172596696 conn(0x7ff6f806b400 msgr2=0x7ff6f815c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 shutdown_connections 2024-09-13T16:53:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:34.135+0000 7ff6f7e00700 1 -- 172.21.15.3:0/3172596696 wait complete. 2024-09-13T16:53:34.145 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:34.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:34.351 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":51,"msgr_send_messages":31,"msgr_recv_bytes":24718,"msgr_send_bytes":29322,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.023472411,"msgr_running_send_time":0.005803777,"msgr_running_recv_time":0.008944933,"msgr_running_fast_dispatch_time":0.002558886,"msgr_send_messages_queue_lat":{"avgcount":30,"sum":0.010346982,"avgtime":0.000344899},"msgr_handle_ack_lat":{"avgcount":36,"sum":0.000010806,"avgtime":0.000000300},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":54,"msgr_send_messages":49,"msgr_recv_bytes":119468,"msgr_send_bytes":22943,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.014540445,"msgr_running_send_time":0.003286051,"msgr_running_recv_time":0.016386946,"msgr_running_fast_dispatch_time":0.001035424,"msgr_send_messages_queue_lat":{"avgcount":49,"sum":0.011517350,"avgtime":0.000235047},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":104976,"msgr_send_encrypted_bytes":8592},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":16,"msgr_recv_bytes":3670,"msgr_send_bytes":68660,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006416889,"msgr_running_send_time":0.001311394,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":15,"sum":0.005825790,"avgtime":0.000388386},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":65984},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.007557903,"avgtime":0.000139961}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199839,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.013203931,"avgtime":0.003300982},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":47320,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":971,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238864,"buffer_anon_items":74,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55215,"mds_co_items":234,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5458,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.131997652,"avgtime":0.002538416},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":109,"get_sum":135162,"get_or_fail_fail":0,"get_or_fail_success":109,"take":0,"take_sum":0,"put":109,"put_sum":135162,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6345,"put":52,"put_sum":6345,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4384,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4384,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:34.353 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 36, "avgtime": 3e-07, "sum": 1.0806e-05 }, "msgr_recv_bytes": 24718, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 51, "msgr_running_fast_dispatch_time": 0.002558886, "msgr_running_recv_time": 0.008944933, "msgr_running_send_time": 0.005803777, "msgr_running_total_time": 0.023472411, "msgr_send_bytes": 29322, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 31, "msgr_send_messages_queue_lat": { "avgcount": 30, "avgtime": 0.000344899, "sum": 0.010346982 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 119468, "msgr_recv_encrypted_bytes": 104976, "msgr_recv_messages": 54, "msgr_running_fast_dispatch_time": 0.001035424, "msgr_running_recv_time": 0.016386946, "msgr_running_send_time": 0.003286051, "msgr_running_total_time": 0.014540445, "msgr_send_bytes": 22943, "msgr_send_encrypted_bytes": 8592, "msgr_send_messages": 49, "msgr_send_messages_queue_lat": { "avgcount": 49, "avgtime": 0.000235047, "sum": 0.01151735 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.001311394, "msgr_running_total_time": 0.006416889, "msgr_send_bytes": 68660, "msgr_send_encrypted_bytes": 65984, "msgr_send_messages": 16, "msgr_send_messages_queue_lat": { "avgcount": 15, "avgtime": 0.000388386, "sum": 0.00582579 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000139961, "sum": 0.007557903 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.003300982, "sum": 0.013203931 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4199839 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 47320 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 971, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238864, "buffer_anon_items": 74, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55215, "mds_co_items": 234, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.002538416, "sum": 0.131997652 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5458, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 109, "get_or_fail_fail": 0, "get_or_fail_success": 109, "get_started": 0, "get_sum": 135162, "max": 104857600, "put": 109, "put_sum": 135162, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6345, "take": 52, "take_sum": 6345, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4384, "max": 3758096384, "put": 4, "put_sum": 4384, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:34.353 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T16:53:39.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:39.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccfe00700 1 Processor -- start 2024-09-13T16:53:39.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccfe00700 1 -- start start 2024-09-13T16:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccfe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9cd006ba80 0x7f9cd006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccfe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9cd0070dc0 0x7f9cd01646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccfe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd00ce5d0 con 0x7f9cd006be60 2024-09-13T16:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccfe00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd015af00 con 0x7f9cd006ba80 2024-09-13T16:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccfe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00cd9c0 con 0x7f9cd0070dc0 2024-09-13T16:53:39.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccea00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9cd006ba80 0x7f9cd006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:39.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccea00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9cd006ba80 0x7f9cd006d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57666/0 (socket says 172.21.15.3:57666) 2024-09-13T16:53:39.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.531+0000 7f9ccea00700 1 -- 172.21.15.3:0/2152455790 learned_addr learned my addr 172.21.15.3:0/2152455790 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:39.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccf400700 1 --2- 172.21.15.3:0/2152455790 >> v2:172.21.15.157:3300/0 conn(0x7f9cd0070dc0 0x7f9cd01646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccea00700 1 -- 172.21.15.3:0/2152455790 >> v2:172.21.15.157:3300/0 conn(0x7f9cd0070dc0 msgr2=0x7f9cd01646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccea00700 1 --2- 172.21.15.3:0/2152455790 >> v2:172.21.15.157:3300/0 conn(0x7f9cd0070dc0 0x7f9cd01646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccea00700 1 -- 172.21.15.3:0/2152455790 >> v1:172.21.15.157:6789/0 conn(0x7f9cd006be60 legacy=0x7f9cd006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccea00700 1 -- 172.21.15.3:0/2152455790 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00cdca0 con 0x7f9cd006ba80 2024-09-13T16:53:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccea00700 1 --2- 172.21.15.3:0/2152455790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd006ba80 0x7f9cd006d6b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0072210 tx=0x7f9cc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965c87f65054d688 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccd600700 1 -- 172.21.15.3:0/2152455790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000e070 con 0x7f9cd006ba80 2024-09-13T16:53:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccd600700 1 -- 172.21.15.3:0/2152455790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc000b9c0 con 0x7f9cd006ba80 2024-09-13T16:53:39.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccd600700 1 -- 172.21.15.3:0/2152455790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000bd70 con 0x7f9cd006ba80 2024-09-13T16:53:39.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2152455790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd006ba80 msgr2=0x7f9cd006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:39.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2152455790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd006ba80 0x7f9cd006d6b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0072210 tx=0x7f9cc000b330 comp rx=0 tx=0).stop 2024-09-13T16:53:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2152455790 shutdown_connections 2024-09-13T16:53:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2152455790 >> v2:172.21.15.157:3300/0 conn(0x7f9cd0070dc0 0x7f9cd01646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2152455790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd006ba80 0x7f9cd006d6b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2152455790 >> 172.21.15.3:0/2152455790 conn(0x7f9cd006a2f0 msgr2=0x7f9cd006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2152455790 shutdown_connections 2024-09-13T16:53:39.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.535+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2152455790 wait complete. 2024-09-13T16:53:39.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.539+0000 7f9ccfe00700 1 Processor -- start 2024-09-13T16:53:39.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.539+0000 7f9ccfe00700 1 -- start start 2024-09-13T16:53:39.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.539+0000 7f9ccfe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd006ba80 0x7f9cd0152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:39.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.539+0000 7f9ccfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd006be60 0x7f9cd0153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:39.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.539+0000 7f9ccfe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0070dc0 0x7f9cd0156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:39.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.539+0000 7f9ccfe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0159ad0 con 0x7f9cd0070dc0 2024-09-13T16:53:39.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.539+0000 7f9ccfe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0153230 con 0x7f9cd006be60 2024-09-13T16:53:39.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.539+0000 7f9ccfe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0154440 con 0x7f9cd006ba80 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0070dc0 0x7f9cd0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0070dc0 0x7f9cd0156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57668/0 (socket says 172.21.15.3:57668) 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 -- 172.21.15.3:0/2581928474 learned_addr learned my addr 172.21.15.3:0/2581928474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccea00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd006ba80 0x7f9cd0152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9cce000700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd006be60 0x7f9cd0153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 -- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd006ba80 msgr2=0x7f9cd0152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd006ba80 0x7f9cd0152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 -- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd006be60 msgr2=0x7f9cd0153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd006be60 0x7f9cd0153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 -- 172.21.15.3:0/2581928474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc0009b30 con 0x7f9cd0070dc0 2024-09-13T16:53:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccf400700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0070dc0 0x7f9cd0156430 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800d580 tx=0x7f9cc800d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9cc6a00700 1 -- 172.21.15.3:0/2581928474 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80125f0 con 0x7f9cd0070dc0 2024-09-13T16:53:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9cc6a00700 1 -- 172.21.15.3:0/2581928474 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc8012750 con 0x7f9cd0070dc0 2024-09-13T16:53:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9cc6a00700 1 -- 172.21.15.3:0/2581928474 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8012b00 con 0x7f9cd0070dc0 2024-09-13T16:53:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd0154720 con 0x7f9cd0070dc0 2024-09-13T16:53:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd0154ce0 con 0x7f9cd0070dc0 2024-09-13T16:53:39.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.543+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd006c2a0 con 0x7f9cd0070dc0 2024-09-13T16:53:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.547+0000 7f9cc6a00700 1 -- 172.21.15.3:0/2581928474 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9cc8012c60 con 0x7f9cd0070dc0 2024-09-13T16:53:39.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.547+0000 7f9cc6a00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9cb0066730 0x7f9cb0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:39.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.547+0000 7f9cc6a00700 1 -- 172.21.15.3:0/2581928474 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f9cc8084480 con 0x7f9cd0070dc0 2024-09-13T16:53:39.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.547+0000 7f9ccea00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9cb0066730 0x7f9cb0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:39.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.551+0000 7f9ccea00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9cb0066730 0x7f9cb0068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0000f80 tx=0x7f9cc0009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:39.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.555+0000 7f9cc6a00700 1 -- 172.21.15.3:0/2581928474 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9cc80515b0 con 0x7f9cd0070dc0 2024-09-13T16:53:39.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.787+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cd00cbaf0 con 0x7f9cd0070dc0 2024-09-13T16:53:39.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.787+0000 7f9cc6a00700 1 -- 172.21.15.3:0/2581928474 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f9cc8054bd0 con 0x7f9cd0070dc0 2024-09-13T16:53:39.795 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:39.795 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:39.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9cb0066730 msgr2=0x7f9cb0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:39.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9cb0066730 0x7f9cb0068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0000f80 tx=0x7f9cc0009000 comp rx=0 tx=0).stop 2024-09-13T16:53:39.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0070dc0 msgr2=0x7f9cd0156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:39.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0070dc0 0x7f9cd0156430 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800d580 tx=0x7f9cc800d940 comp rx=0 tx=0).stop 2024-09-13T16:53:39.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 shutdown_connections 2024-09-13T16:53:39.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f9cb0066730 0x7f9cb0068bf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0070dc0 0x7f9cd0156430 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd006be60 0x7f9cd0153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 --2- 172.21.15.3:0/2581928474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd006ba80 0x7f9cd0152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:39.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 >> 172.21.15.3:0/2581928474 conn(0x7f9cd006a2f0 msgr2=0x7f9cd00ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:39.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.791+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 shutdown_connections 2024-09-13T16:53:39.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:39.795+0000 7f9ccfe00700 1 -- 172.21.15.3:0/2581928474 wait complete. 2024-09-13T16:53:39.802 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:39.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:40.005 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":58,"msgr_send_messages":32,"msgr_recv_bytes":27496,"msgr_send_bytes":29440,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.025795442,"msgr_running_send_time":0.007068047,"msgr_running_recv_time":0.009496731,"msgr_running_fast_dispatch_time":0.003064961,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.010422772,"avgtime":0.000336218},"msgr_handle_ack_lat":{"avgcount":44,"sum":0.000013168,"avgtime":0.000000299},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":56,"msgr_send_messages":51,"msgr_recv_bytes":120012,"msgr_send_bytes":24127,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.014880737,"msgr_running_send_time":0.003454255,"msgr_running_recv_time":0.016604177,"msgr_running_fast_dispatch_time":0.001035424,"msgr_send_messages_queue_lat":{"avgcount":51,"sum":0.011733216,"avgtime":0.000230063},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":105520,"msgr_send_encrypted_bytes":9776},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":17,"msgr_recv_bytes":3670,"msgr_send_bytes":70436,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006528752,"msgr_running_send_time":0.001399570,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":16,"sum":0.005929657,"avgtime":0.000370603},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":67760},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.007557903,"avgtime":0.000139961}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199839,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.013203931,"avgtime":0.003300982},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":47704,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":2,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":976,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238864,"buffer_anon_items":74,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55215,"mds_co_items":234,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5458,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.131997652,"avgtime":0.002538416},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":118,"get_sum":137570,"get_or_fail_fail":0,"get_or_fail_success":118,"take":0,"take_sum":0,"put":118,"put_sum":137570,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6345,"put":52,"put_sum":6345,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4384,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4384,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:40.007 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 44, "avgtime": 2.99e-07, "sum": 1.3168e-05 }, "msgr_recv_bytes": 27496, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 58, "msgr_running_fast_dispatch_time": 0.003064961, "msgr_running_recv_time": 0.009496731, "msgr_running_send_time": 0.007068047, "msgr_running_total_time": 0.025795442, "msgr_send_bytes": 29440, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 32, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.000336218, "sum": 0.010422772 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 120012, "msgr_recv_encrypted_bytes": 105520, "msgr_recv_messages": 56, "msgr_running_fast_dispatch_time": 0.001035424, "msgr_running_recv_time": 0.016604177, "msgr_running_send_time": 0.003454255, "msgr_running_total_time": 0.014880737, "msgr_send_bytes": 24127, "msgr_send_encrypted_bytes": 9776, "msgr_send_messages": 51, "msgr_send_messages_queue_lat": { "avgcount": 51, "avgtime": 0.000230063, "sum": 0.011733216 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.00139957, "msgr_running_total_time": 0.006528752, "msgr_send_bytes": 70436, "msgr_send_encrypted_bytes": 67760, "msgr_send_messages": 17, "msgr_send_messages_queue_lat": { "avgcount": 16, "avgtime": 0.000370603, "sum": 0.005929657 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000139961, "sum": 0.007557903 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.003300982, "sum": 0.013203931 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4199839 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 47704 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 2, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 976, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238864, "buffer_anon_items": 74, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55215, "mds_co_items": 234, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.002538416, "sum": 0.131997652 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5458, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 118, "get_or_fail_fail": 0, "get_or_fail_success": 118, "get_started": 0, "get_sum": 137570, "max": 104857600, "put": 118, "put_sum": 137570, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6345, "take": 52, "take_sum": 6345, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4384, "max": 3758096384, "put": 4, "put_sum": 4384, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:40.007 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T16:53:45.008 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.179+0000 7fe855a00700 1 Processor -- start 2024-09-13T16:53:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.179+0000 7fe855a00700 1 -- start start 2024-09-13T16:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe855a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe85015cc40 0x7fe85006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe855a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe85006d540 0x7fe8500ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe855a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe8500ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe855a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8500adc60 con 0x7fe85006d540 2024-09-13T16:53:45.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe855a00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85006da50 con 0x7fe85015cc40 2024-09-13T16:53:45.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe855a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85006dd30 con 0x7fe85015d020 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe85006d540 0x7fe8500ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe85006d540 0x7fe8500ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49220/0 (socket says 172.21.15.3:49220) 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 -- 172.21.15.3:0/3967007691 learned_addr learned my addr 172.21.15.3:0/3967007691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe84fe00700 1 --2- 172.21.15.3:0/3967007691 >> v2:172.21.15.157:3300/0 conn(0x7fe85015cc40 0x7fe85006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe855000700 1 --2- 172.21.15.3:0/3967007691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe8500ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 -- 172.21.15.3:0/3967007691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 msgr2=0x7fe8500ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 --2- 172.21.15.3:0/3967007691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe8500ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 -- 172.21.15.3:0/3967007691 >> v2:172.21.15.157:3300/0 conn(0x7fe85015cc40 msgr2=0x7fe85006fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 --2- 172.21.15.3:0/3967007691 >> v2:172.21.15.157:3300/0 conn(0x7fe85015cc40 0x7fe85006fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 -- 172.21.15.3:0/3967007691 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8500ad610 con 0x7fe85006d540 2024-09-13T16:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe847400700 1 --2- 172.21.15.3:0/3967007691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500ac8d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe850067fd0 tx=0x7fe838009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec45e32c0f8697d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe84f400700 1 -- 172.21.15.3:0/3967007691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838011830 con 0x7fe85006d540 2024-09-13T16:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe84f400700 1 -- 172.21.15.3:0/3967007691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838011990 con 0x7fe85006d540 2024-09-13T16:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.183+0000 7fe84f400700 1 -- 172.21.15.3:0/3967007691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838011d20 con 0x7fe85006d540 2024-09-13T16:53:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 -- 172.21.15.3:0/3967007691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 msgr2=0x7fe8500ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 --2- 172.21.15.3:0/3967007691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500ac8d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe850067fd0 tx=0x7fe838009bf0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 -- 172.21.15.3:0/3967007691 shutdown_connections 2024-09-13T16:53:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 --2- 172.21.15.3:0/3967007691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500ac8d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 --2- 172.21.15.3:0/3967007691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe8500ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 --2- 172.21.15.3:0/3967007691 >> v2:172.21.15.157:3300/0 conn(0x7fe85015cc40 0x7fe85006fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 -- 172.21.15.3:0/3967007691 >> 172.21.15.3:0/3967007691 conn(0x7fe85006a2f0 msgr2=0x7fe85006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 -- 172.21.15.3:0/3967007691 shutdown_connections 2024-09-13T16:53:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 -- 172.21.15.3:0/3967007691 wait complete. 2024-09-13T16:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 Processor -- start 2024-09-13T16:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.187+0000 7fe855a00700 1 -- start start 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85015cc40 0x7fe8501533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe850153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe850155060 con 0x7fe85006d540 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500ccb10 con 0x7fe85015cc40 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500ccdf0 con 0x7fe85015d020 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49222/0 (socket says 172.21.15.3:49222) 2024-09-13T16:53:45.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 -- 172.21.15.3:0/1001712272 learned_addr learned my addr 172.21.15.3:0/1001712272 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe847400700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85015cc40 0x7fe8501533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855000700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe850153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 -- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 msgr2=0x7fe850153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe850153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 -- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85015cc40 msgr2=0x7fe8501533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85015cc40 0x7fe8501533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 -- 172.21.15.3:0/1001712272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838009570 con 0x7fe85006d540 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855000700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe850153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84fe00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500cc050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e940 tx=0x7fe84000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84cc00700 1 -- 172.21.15.3:0/1001712272 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84000ee90 con 0x7fe85006d540 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84cc00700 1 -- 172.21.15.3:0/1001712272 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84000c850 con 0x7fe85006d540 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8500cd0d0 con 0x7fe85006d540 2024-09-13T16:53:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe84cc00700 1 -- 172.21.15.3:0/1001712272 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84000cbe0 con 0x7fe85006d540 2024-09-13T16:53:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.191+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8500cd690 con 0x7fe85006d540 2024-09-13T16:53:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.195+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85006fc20 con 0x7fe85006d540 2024-09-13T16:53:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.195+0000 7fe84cc00700 1 -- 172.21.15.3:0/1001712272 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe84000cd40 con 0x7fe85006d540 2024-09-13T16:53:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.195+0000 7fe84cc00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe82c066730 0x7fe82c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.195+0000 7fe84cc00700 1 -- 172.21.15.3:0/1001712272 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fe8400844c0 con 0x7fe85006d540 2024-09-13T16:53:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.199+0000 7fe847400700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe82c066730 0x7fe82c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:45.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.199+0000 7fe847400700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe82c066730 0x7fe82c068bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe838000c00 tx=0x7fe838008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:45.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.203+0000 7fe84cc00700 1 -- 172.21.15.3:0/1001712272 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe8400514e0 con 0x7fe85006d540 2024-09-13T16:53:45.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.431+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8500708d0 con 0x7fe85006d540 2024-09-13T16:53:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe84cc00700 1 -- 172.21.15.3:0/1001712272 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fe840054b00 con 0x7fe85006d540 2024-09-13T16:53:45.441 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:45.441 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe82c066730 msgr2=0x7fe82c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe82c066730 0x7fe82c068bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe838000c00 tx=0x7fe838008030 comp rx=0 tx=0).stop 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 msgr2=0x7fe8500cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500cc050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e940 tx=0x7fe84000ec50 comp rx=0 tx=0).stop 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 shutdown_connections 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe82c066730 0x7fe82c068bf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe85015d020 0x7fe850153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85015cc40 0x7fe8501533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 --2- 172.21.15.3:0/1001712272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe85006d540 0x7fe8500cc050 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.435+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 >> 172.21.15.3:0/1001712272 conn(0x7fe85006a2f0 msgr2=0x7fe85006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.439+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 shutdown_connections 2024-09-13T16:53:45.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:45.439+0000 7fe855a00700 1 -- 172.21.15.3:0/1001712272 wait complete. 2024-09-13T16:53:45.446 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:45.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:45.663 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":63,"msgr_send_messages":32,"msgr_recv_bytes":29586,"msgr_send_bytes":29440,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.027351512,"msgr_running_send_time":0.007850566,"msgr_running_recv_time":0.009837954,"msgr_running_fast_dispatch_time":0.003472311,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.010422772,"avgtime":0.000336218},"msgr_handle_ack_lat":{"avgcount":49,"sum":0.000014265,"avgtime":0.000000291},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":57,"msgr_send_messages":52,"msgr_recv_bytes":120284,"msgr_send_bytes":24719,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.015164881,"msgr_running_send_time":0.003598899,"msgr_running_recv_time":0.016724146,"msgr_running_fast_dispatch_time":0.001035424,"msgr_send_messages_queue_lat":{"avgcount":52,"sum":0.011856587,"avgtime":0.000228011},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":105792,"msgr_send_encrypted_bytes":10368},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":18,"msgr_recv_bytes":3670,"msgr_send_bytes":72212,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006623294,"msgr_running_send_time":0.001475284,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":17,"sum":0.005991644,"avgtime":0.000352449},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":69536},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.007557903,"avgtime":0.000139961}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199839,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.013203931,"avgtime":0.003300982},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":48088,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":2,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":981,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238864,"buffer_anon_items":74,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55215,"mds_co_items":234,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5458,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.131997652,"avgtime":0.002538416},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":124,"get_sum":139340,"get_or_fail_fail":0,"get_or_fail_success":124,"take":0,"take_sum":0,"put":124,"put_sum":139340,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6345,"put":52,"put_sum":6345,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4384,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4384,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:45.664 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 49, "avgtime": 2.91e-07, "sum": 1.4265e-05 }, "msgr_recv_bytes": 29586, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 63, "msgr_running_fast_dispatch_time": 0.003472311, "msgr_running_recv_time": 0.009837954, "msgr_running_send_time": 0.007850566, "msgr_running_total_time": 0.027351512, "msgr_send_bytes": 29440, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 32, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.000336218, "sum": 0.010422772 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 120284, "msgr_recv_encrypted_bytes": 105792, "msgr_recv_messages": 57, "msgr_running_fast_dispatch_time": 0.001035424, "msgr_running_recv_time": 0.016724146, "msgr_running_send_time": 0.003598899, "msgr_running_total_time": 0.015164881, "msgr_send_bytes": 24719, "msgr_send_encrypted_bytes": 10368, "msgr_send_messages": 52, "msgr_send_messages_queue_lat": { "avgcount": 52, "avgtime": 0.000228011, "sum": 0.011856587 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.001475284, "msgr_running_total_time": 0.006623294, "msgr_send_bytes": 72212, "msgr_send_encrypted_bytes": 69536, "msgr_send_messages": 18, "msgr_send_messages_queue_lat": { "avgcount": 17, "avgtime": 0.000352449, "sum": 0.005991644 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000139961, "sum": 0.007557903 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.003300982, "sum": 0.013203931 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4199839 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 48088 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 2, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 981, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238864, "buffer_anon_items": 74, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55215, "mds_co_items": 234, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.002538416, "sum": 0.131997652 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5458, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 124, "get_or_fail_fail": 0, "get_or_fail_success": 124, "get_started": 0, "get_sum": 139340, "max": 104857600, "put": 124, "put_sum": 139340, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6345, "take": 52, "take_sum": 6345, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4384, "max": 3758096384, "put": 4, "put_sum": 4384, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:45.664 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T16:53:50.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.831+0000 7f2af8c00700 1 Processor -- start 2024-09-13T16:53:50.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.831+0000 7f2af8c00700 1 -- start start 2024-09-13T16:53:50.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af8c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af41652e0 0x7f2af416bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:50.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af8c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af406ba80 con 0x7f2af4160f30 2024-09-13T16:53:50.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af8c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af406fad0 con 0x7f2af4165770 2024-09-13T16:53:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af8c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af406bcd0 con 0x7f2af41652e0 2024-09-13T16:53:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2af4160f30 0x7f2af406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57770/0 (socket says 172.21.15.3:57770) 2024-09-13T16:53:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3400700 1 -- 172.21.15.3:0/1242592625 learned_addr learned my addr 172.21.15.3:0/1242592625 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3e00700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af41652e0 0x7f2af416bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425888429 0 0) 0x7f2af406ba80 con 0x7f2af4160f30 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad80035d0 con 0x7f2af4160f30 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387496407 0 0) 0x7f2af406fad0 con 0x7f2af4165770 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af406ba80 con 0x7f2af4165770 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3608228061 0 0) 0x7f2ad80035d0 con 0x7f2af4160f30 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af41652e0 msgr2=0x7f2af416bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af41652e0 0x7f2af416bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 >> v1:172.21.15.157:6789/0 conn(0x7f2af4165770 legacy=0x7f2af4164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af406bfb0 con 0x7f2af4160f30 2024-09-13T16:53:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7f2ae4001670 con 0x7f2af4160f30 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 >> v1:172.21.15.3:6789/0 conn(0x7f2af4160f30 legacy=0x7f2af406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2ad8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af41652e0 0x7f2ad8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad8004db0 0x7f2ad8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af406bcd0 con 0x7f2ad8003be0 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad800bae0 con 0x7f2ad8004db0 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad800bc50 con 0x7f2af41652e0 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3400700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2ad8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3e00700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad8004db0 0x7f2ad8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af2a00700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af41652e0 0x7f2ad8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:50.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3400700 1 -- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af41652e0 msgr2=0x7f2ad8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:50.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3400700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af41652e0 0x7f2ad8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:50.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3400700 1 -- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad8004db0 msgr2=0x7f2ad8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:50.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3400700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad8004db0 0x7f2ad8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:50.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.835+0000 7f2af3400700 1 -- 172.21.15.3:0/1242592625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad80035d0 con 0x7f2ad8003be0 2024-09-13T16:53:50.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af3400700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2ad8003fe0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c210 tx=0x7f2ad80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591fc9c38770bf07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:50.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400cdb0 con 0x7f2ad8003be0 2024-09-13T16:53:50.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae4005980 con 0x7f2ad8003be0 2024-09-13T16:53:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af2000700 1 -- 172.21.15.3:0/1242592625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4005d30 con 0x7f2ad8003be0 2024-09-13T16:53:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af8c00700 1 -- 172.21.15.3:0/1242592625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 msgr2=0x7f2ad8003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af8c00700 1 --2- 172.21.15.3:0/1242592625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2ad8003fe0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c210 tx=0x7f2ad80058a0 comp rx=0 tx=0).stop 2024-09-13T16:53:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af3e00700 1 -- 172.21.15.3:0/1242592625 reap_dead start 2024-09-13T16:53:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af8c00700 1 -- 172.21.15.3:0/1242592625 shutdown_connections 2024-09-13T16:53:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af8c00700 1 -- 172.21.15.3:0/1242592625 >> 172.21.15.3:0/1242592625 conn(0x7f2af406a2f0 msgr2=0x7f2af406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af8c00700 1 -- 172.21.15.3:0/1242592625 shutdown_connections 2024-09-13T16:53:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.839+0000 7f2af8c00700 1 -- 172.21.15.3:0/1242592625 wait complete. 2024-09-13T16:53:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af8c00700 1 Processor -- start 2024-09-13T16:53:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af8c00700 1 -- start start 2024-09-13T16:53:50.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af8c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2af40a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:50.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af8c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af4161610 0x7f2af4163f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:50.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af8c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af4155890 0x7f2af4143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:50.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af8c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4164470 con 0x7f2ad8003be0 2024-09-13T16:53:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af8c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4141950 con 0x7f2af4155890 2024-09-13T16:53:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af8c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4141c30 con 0x7f2af4161610 2024-09-13T16:53:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2af40a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2af40a85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49236/0 (socket says 172.21.15.3:49236) 2024-09-13T16:53:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3400700 1 -- 172.21.15.3:0/1033068164 learned_addr learned my addr 172.21.15.3:0/1033068164 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3e00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af4155890 0x7f2af4143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af2a00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af4161610 0x7f2af4163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3400700 1 -- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af4161610 msgr2=0x7f2af4163f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3400700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af4161610 0x7f2af4163f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3400700 1 -- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af4155890 msgr2=0x7f2af4143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3400700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af4155890 0x7f2af4143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af3400700 1 -- 172.21.15.3:0/1033068164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af4141f10 con 0x7f2ad8003be0 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.843+0000 7f2af2a00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af4161610 0x7f2af4163f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.847+0000 7f2af3400700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2af40a85c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c800 tx=0x7f2ae4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.847+0000 7f2aeb400700 1 -- 172.21.15.3:0/1033068164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4007bb0 con 0x7f2ad8003be0 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.847+0000 7f2aeb400700 1 -- 172.21.15.3:0/1033068164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae4004890 con 0x7f2ad8003be0 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.847+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af4164470 con 0x7f2ad8003be0 2024-09-13T16:53:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.847+0000 7f2aeb400700 1 -- 172.21.15.3:0/1033068164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4004c00 con 0x7f2ad8003be0 2024-09-13T16:53:50.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.847+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af41424d0 con 0x7f2ad8003be0 2024-09-13T16:53:50.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.847+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af4164bd0 con 0x7f2ad8003be0 2024-09-13T16:53:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.851+0000 7f2aeb400700 1 -- 172.21.15.3:0/1033068164 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ae40070d0 con 0x7f2ad8003be0 2024-09-13T16:53:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.851+0000 7f2aeb400700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad4066730 0x7f2ad4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:50.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.851+0000 7f2aeb400700 1 -- 172.21.15.3:0/1033068164 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f2ae4085bf0 con 0x7f2ad8003be0 2024-09-13T16:53:50.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.851+0000 7f2af2a00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad4066730 0x7f2ad4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.855+0000 7f2af2a00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad4066730 0x7f2ad4068bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2af4156b40 tx=0x7f2ae0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:50.859+0000 7f2aeb400700 1 -- 172.21.15.3:0/1033068164 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2ae4052e40 con 0x7f2ad8003be0 2024-09-13T16:53:51.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.083+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2af40b2e60 con 0x7f2ad8003be0 2024-09-13T16:53:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.087+0000 7f2aeb400700 1 -- 172.21.15.3:0/1033068164 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f2ae4025e80 con 0x7f2ad8003be0 2024-09-13T16:53:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:51.093 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad4066730 msgr2=0x7f2ad4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad4066730 0x7f2ad4068bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2af4156b40 tx=0x7f2ae0009040 comp rx=0 tx=0).stop 2024-09-13T16:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 msgr2=0x7f2af40a85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2af40a85c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c800 tx=0x7f2ae4001980 comp rx=0 tx=0).stop 2024-09-13T16:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 shutdown_connections 2024-09-13T16:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2ad4066730 0x7f2ad4068bf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af4155890 0x7f2af4143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2af4161610 0x7f2af4163f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 --2- 172.21.15.3:0/1033068164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ad8003be0 0x7f2af40a85c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 >> 172.21.15.3:0/1033068164 conn(0x7f2af406a2f0 msgr2=0x7f2af40ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 shutdown_connections 2024-09-13T16:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:51.091+0000 7f2af8c00700 1 -- 172.21.15.3:0/1033068164 wait complete. 2024-09-13T16:53:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:51.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:51.300 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":70,"msgr_send_messages":32,"msgr_recv_bytes":32216,"msgr_send_bytes":29440,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.029603793,"msgr_running_send_time":0.009044994,"msgr_running_recv_time":0.010333561,"msgr_running_fast_dispatch_time":0.004053553,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.010422772,"avgtime":0.000336218},"msgr_handle_ack_lat":{"avgcount":56,"sum":0.000015907,"avgtime":0.000000284},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":58,"msgr_send_messages":53,"msgr_recv_bytes":120556,"msgr_send_bytes":25311,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.015295523,"msgr_running_send_time":0.003670723,"msgr_running_recv_time":0.016795750,"msgr_running_fast_dispatch_time":0.001035424,"msgr_send_messages_queue_lat":{"avgcount":53,"sum":0.011919238,"avgtime":0.000224891},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":106064,"msgr_send_encrypted_bytes":10960},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":19,"msgr_recv_bytes":3670,"msgr_send_bytes":73988,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006728825,"msgr_running_send_time":0.001560423,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":18,"sum":0.006121338,"avgtime":0.000340074},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":71312},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.007557903,"avgtime":0.000139961}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":1,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":5,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4202044,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.013203931,"avgtime":0.003300982},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":2,"cap-":1,"rss":48344,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":2,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":986,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238864,"buffer_anon_items":74,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55167,"mds_co_items":233,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5458,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.131997652,"avgtime":0.002538416},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":132,"get_sum":141470,"get_or_fail_fail":0,"get_or_fail_success":132,"take":0,"take_sum":0,"put":132,"put_sum":141470,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6345,"put":52,"put_sum":6345,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":2205,"max":3758096384,"get_started":0,"get":5,"get_sum":6589,"get_or_fail_fail":0,"get_or_fail_success":5,"take":0,"take_sum":0,"put":4,"put_sum":4384,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:51.303 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 56, "avgtime": 2.84e-07, "sum": 1.5907e-05 }, "msgr_recv_bytes": 32216, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 70, "msgr_running_fast_dispatch_time": 0.004053553, "msgr_running_recv_time": 0.010333561, "msgr_running_send_time": 0.009044994, "msgr_running_total_time": 0.029603793, "msgr_send_bytes": 29440, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 32, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.000336218, "sum": 0.010422772 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 120556, "msgr_recv_encrypted_bytes": 106064, "msgr_recv_messages": 58, "msgr_running_fast_dispatch_time": 0.001035424, "msgr_running_recv_time": 0.01679575, "msgr_running_send_time": 0.003670723, "msgr_running_total_time": 0.015295523, "msgr_send_bytes": 25311, "msgr_send_encrypted_bytes": 10960, "msgr_send_messages": 53, "msgr_send_messages_queue_lat": { "avgcount": 53, "avgtime": 0.000224891, "sum": 0.011919238 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.001560423, "msgr_running_total_time": 0.006728825, "msgr_send_bytes": 73988, "msgr_send_encrypted_bytes": 71312, "msgr_send_messages": 19, "msgr_send_messages_queue_lat": { "avgcount": 18, "avgtime": 0.000340074, "sum": 0.006121338 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000139961, "sum": 0.007557903 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 5, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.003300982, "sum": 0.013203931 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4202044 }, "mds_mem": { "cap": 1, "cap+": 2, "cap-": 1, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 48344 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 2, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 986, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238864, "buffer_anon_items": 74, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55167, "mds_co_items": 233, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.002538416, "sum": 0.131997652 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5458, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 132, "get_or_fail_fail": 0, "get_or_fail_success": 132, "get_started": 0, "get_sum": 141470, "max": 104857600, "put": 132, "put_sum": 141470, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6345, "take": 52, "take_sum": 6345, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 5, "get_or_fail_fail": 0, "get_or_fail_success": 5, "get_started": 0, "get_sum": 6589, "max": 3758096384, "put": 4, "put_sum": 4384, "take": 0, "take_sum": 0, "val": 2205, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:51.303 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T16:53:56.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:56.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.482+0000 7fcd07000700 1 Processor -- start 2024-09-13T16:53:56.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.482+0000 7fcd07000700 1 -- start start 2024-09-13T16:53:56.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fcd07000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd000b2ba0 0x7fcd00167cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fcd07000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd000cd130 con 0x7fcd00071e60 2024-09-13T16:53:56.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fcd07000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd000cd3d0 con 0x7fcd000ce470 2024-09-13T16:53:56.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fcd07000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000cd6b0 con 0x7fcd000b2ba0 2024-09-13T16:53:56.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fccffe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcd00071e60 0x7fcd0015dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42576/0 (socket says 172.21.15.3:42576) 2024-09-13T16:53:56.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fccffe00700 1 -- 172.21.15.3:0/905201908 learned_addr learned my addr 172.21.15.3:0/905201908 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:56.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fcd05000700 1 --2- 172.21.15.3:0/905201908 >> v2:172.21.15.157:3300/0 conn(0x7fcd000b2ba0 0x7fcd00167cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fccff400700 1 -- 172.21.15.3:0/905201908 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533143350 0 0) 0x7fcd000cd130 con 0x7fcd00071e60 2024-09-13T16:53:56.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fccff400700 1 -- 172.21.15.3:0/905201908 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce40035d0 con 0x7fcd00071e60 2024-09-13T16:53:56.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fccff400700 1 -- 172.21.15.3:0/905201908 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1079439991 0 0) 0x7fcce40035d0 con 0x7fcd00071e60 2024-09-13T16:53:56.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fcd05000700 1 -- 172.21.15.3:0/905201908 >> v1:172.21.15.157:6789/0 conn(0x7fcd000ce470 legacy=0x7fcd000b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fcd05000700 1 -- 172.21.15.3:0/905201908 >> v1:172.21.15.3:6789/0 conn(0x7fcd00071e60 legacy=0x7fcd0015dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.486+0000 7fcd05000700 1 -- 172.21.15.3:0/905201908 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000cd990 con 0x7fcd000b2ba0 2024-09-13T16:53:56.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fcd05000700 1 --2- 172.21.15.3:0/905201908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b2ba0 0x7fcd00167cd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009880 tx=0x7fccf8009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe678d8487879b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:56.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fccff400700 1 -- 172.21.15.3:0/905201908 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0001a10 con 0x7fcd000b2ba0 2024-09-13T16:53:56.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fccff400700 1 -- 172.21.15.3:0/905201908 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8004030 con 0x7fcd000b2ba0 2024-09-13T16:53:56.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fccff400700 1 -- 172.21.15.3:0/905201908 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf80043e0 con 0x7fcd000b2ba0 2024-09-13T16:53:56.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fcd07000700 1 -- 172.21.15.3:0/905201908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b2ba0 msgr2=0x7fcd00167cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fcd07000700 1 --2- 172.21.15.3:0/905201908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b2ba0 0x7fcd00167cd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009880 tx=0x7fccf8009b90 comp rx=0 tx=0).stop 2024-09-13T16:53:56.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fcd07000700 1 -- 172.21.15.3:0/905201908 shutdown_connections 2024-09-13T16:53:56.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fcd07000700 1 --2- 172.21.15.3:0/905201908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b2ba0 0x7fcd00167cd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fcd07000700 1 -- 172.21.15.3:0/905201908 >> 172.21.15.3:0/905201908 conn(0x7fcd000b07a0 msgr2=0x7fcd000b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:56.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fcd07000700 1 -- 172.21.15.3:0/905201908 shutdown_connections 2024-09-13T16:53:56.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.490+0000 7fcd07000700 1 -- 172.21.15.3:0/905201908 wait complete. 2024-09-13T16:53:56.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd07000700 1 Processor -- start 2024-09-13T16:53:56.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd07000700 1 -- start start 2024-09-13T16:53:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd07000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd00071e60 0x7fcd0015efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd07000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd000b2ba0 0x7fcd0015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd07000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000ce470 0x7fcd000aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd07000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000abcb0 con 0x7fcd00071e60 2024-09-13T16:53:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd07000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00150050 con 0x7fcd000ce470 2024-09-13T16:53:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd07000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd001f7960 con 0x7fcd000b2ba0 2024-09-13T16:53:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccffe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd00071e60 0x7fcd0015efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccffe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd00071e60 0x7fcd0015efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40240/0 (socket says 172.21.15.3:40240) 2024-09-13T16:53:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccffe00700 1 -- 172.21.15.3:0/2221014106 learned_addr learned my addr 172.21.15.3:0/2221014106 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd05000700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000ce470 0x7fcd000aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccf7400700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd000b2ba0 0x7fcd0015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccffe00700 1 -- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd000b2ba0 msgr2=0x7fcd0015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccffe00700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd000b2ba0 0x7fcd0015f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccffe00700 1 -- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000ce470 msgr2=0x7fcd000aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccffe00700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000ce470 0x7fcd000aa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fccffe00700 1 -- 172.21.15.3:0/2221014106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf80094b0 con 0x7fcd00071e60 2024-09-13T16:53:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.494+0000 7fcd05000700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000ce470 0x7fcd000aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.498+0000 7fccffe00700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd00071e60 0x7fcd0015efd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fccf0001890 tx=0x7fccf000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.498+0000 7fccfcc00700 1 -- 172.21.15.3:0/2221014106 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000ac90 con 0x7fcd00071e60 2024-09-13T16:53:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.498+0000 7fccfcc00700 1 -- 172.21.15.3:0/2221014106 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf000adf0 con 0x7fcd00071e60 2024-09-13T16:53:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.498+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd001f7c40 con 0x7fcd00071e60 2024-09-13T16:53:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.498+0000 7fccfcc00700 1 -- 172.21.15.3:0/2221014106 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0007310 con 0x7fcd00071e60 2024-09-13T16:53:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.498+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd001f8200 con 0x7fcd00071e60 2024-09-13T16:53:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.498+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd001f8420 con 0x7fcd00071e60 2024-09-13T16:53:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.502+0000 7fccfcc00700 1 -- 172.21.15.3:0/2221014106 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccf0007470 con 0x7fcd00071e60 2024-09-13T16:53:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.502+0000 7fccfcc00700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccdc066730 0x7fccdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.502+0000 7fccfcc00700 1 -- 172.21.15.3:0/2221014106 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fccf0083ef0 con 0x7fcd00071e60 2024-09-13T16:53:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.502+0000 7fccf7400700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccdc066730 0x7fccdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.502+0000 7fccf7400700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccdc066730 0x7fccdc068bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcd000a9f20 tx=0x7fcce8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:56.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.510+0000 7fccfcc00700 1 -- 172.21.15.3:0/2221014106 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fccf0051020 con 0x7fcd00071e60 2024-09-13T16:53:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.742+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd000719e0 con 0x7fcd00071e60 2024-09-13T16:53:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fccfcc00700 1 -- 172.21.15.3:0/2221014106 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fccf0018070 con 0x7fcd00071e60 2024-09-13T16:53:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:56.752 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccdc066730 msgr2=0x7fccdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccdc066730 0x7fccdc068bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcd000a9f20 tx=0x7fcce8009040 comp rx=0 tx=0).stop 2024-09-13T16:53:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd00071e60 msgr2=0x7fcd0015efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd00071e60 0x7fcd0015efd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fccf0001890 tx=0x7fccf000a340 comp rx=0 tx=0).stop 2024-09-13T16:53:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 shutdown_connections 2024-09-13T16:53:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccdc066730 0x7fccdc068bf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000ce470 0x7fcd000aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd000b2ba0 0x7fcd0015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 --2- 172.21.15.3:0/2221014106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd00071e60 0x7fcd0015efd0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 >> 172.21.15.3:0/2221014106 conn(0x7fcd000b07a0 msgr2=0x7fcd00070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 shutdown_connections 2024-09-13T16:53:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:56.746+0000 7fcd07000700 1 -- 172.21.15.3:0/2221014106 wait complete. 2024-09-13T16:53:56.757 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:56.781 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:56.977 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":81,"msgr_send_messages":37,"msgr_recv_bytes":36204,"msgr_send_bytes":31410,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.032398325,"msgr_running_send_time":0.010533960,"msgr_running_recv_time":0.010957589,"msgr_running_fast_dispatch_time":0.004687076,"msgr_send_messages_queue_lat":{"avgcount":36,"sum":0.010786817,"avgtime":0.000299633},"msgr_handle_ack_lat":{"avgcount":64,"sum":0.000018244,"avgtime":0.000000285},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":62,"msgr_send_messages":57,"msgr_recv_bytes":121592,"msgr_send_bytes":29412,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.016060060,"msgr_running_send_time":0.004050013,"msgr_running_recv_time":0.017084545,"msgr_running_fast_dispatch_time":0.001114479,"msgr_send_messages_queue_lat":{"avgcount":57,"sum":0.012310684,"avgtime":0.000215976},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":106608,"msgr_send_encrypted_bytes":12144},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":20,"msgr_recv_bytes":3670,"msgr_send_bytes":75764,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006826444,"msgr_running_send_time":0.001638751,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":19,"sum":0.006251870,"avgtime":0.000329045},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":73088},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":59,"sum":0.007982537,"avgtime":0.000135297}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":1,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":5,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4202044,"rdpos":4195196,"jlat":{"avgcount":5,"sum":0.018869506,"avgtime":0.003773901},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":2,"cap-":1,"rss":48600,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":3,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":991,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":254896,"buffer_anon_items":78,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56543,"mds_co_items":238,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":58,"op_send_bytes":8155,"op_resend":0,"op_reply":58,"op_latency":{"avgcount":58,"sum":0.171997174,"avgtime":0.002965468},"op_inflight":0,"oplen_avg":{"avgcount":58,"sum":116},"op":58,"op_r":36,"op_w":22,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":5,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":16,"omap_rd":48,"omap_del":5},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":3,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":147,"get_sum":145040,"get_or_fail_fail":0,"get_or_fail_success":147,"take":0,"take_sum":0,"put":147,"put_sum":145040,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":58,"take_sum":9042,"put":58,"put_sum":9042,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":58,"take_sum":58,"put":58,"put_sum":58,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":5,"get_sum":6589,"get_or_fail_fail":0,"get_or_fail_success":5,"take":0,"take_sum":0,"put":5,"put_sum":6589,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:56.978 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 64, "avgtime": 2.85e-07, "sum": 1.8244e-05 }, "msgr_recv_bytes": 36204, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 81, "msgr_running_fast_dispatch_time": 0.004687076, "msgr_running_recv_time": 0.010957589, "msgr_running_send_time": 0.01053396, "msgr_running_total_time": 0.032398325, "msgr_send_bytes": 31410, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 37, "msgr_send_messages_queue_lat": { "avgcount": 36, "avgtime": 0.000299633, "sum": 0.010786817 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 121592, "msgr_recv_encrypted_bytes": 106608, "msgr_recv_messages": 62, "msgr_running_fast_dispatch_time": 0.001114479, "msgr_running_recv_time": 0.017084545, "msgr_running_send_time": 0.004050013, "msgr_running_total_time": 0.01606006, "msgr_send_bytes": 29412, "msgr_send_encrypted_bytes": 12144, "msgr_send_messages": 57, "msgr_send_messages_queue_lat": { "avgcount": 57, "avgtime": 0.000215976, "sum": 0.012310684 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.001638751, "msgr_running_total_time": 0.006826444, "msgr_send_bytes": 75764, "msgr_send_encrypted_bytes": 73088, "msgr_send_messages": 20, "msgr_send_messages_queue_lat": { "avgcount": 19, "avgtime": 0.000329045, "sum": 0.00625187 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 59, "avgtime": 0.000135297, "sum": 0.007982537 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 5, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.003773901, "sum": 0.018869506 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4202044 }, "mds_mem": { "cap": 1, "cap+": 2, "cap-": 1, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 48600 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 3, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 991, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 254896, "buffer_anon_items": 78, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 56543, "mds_co_items": 238, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 5, "omap_rd": 48, "omap_wr": 16, "op": 58, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 58, "avgtime": 0.002965468, "sum": 0.171997174 }, "op_pg": 0, "op_r": 36, "op_reply": 58, "op_resend": 0, "op_rmw": 0, "op_send": 58, "op_send_bytes": 8155, "op_w": 22, "oplen_avg": { "avgcount": 58, "sum": 116 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 3 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 147, "get_or_fail_fail": 0, "get_or_fail_success": 147, "get_started": 0, "get_sum": 145040, "max": 104857600, "put": 147, "put_sum": 145040, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 58, "put_sum": 9042, "take": 58, "take_sum": 9042, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 58, "put_sum": 58, "take": 58, "take_sum": 58, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 5, "get_or_fail_fail": 0, "get_or_fail_success": 5, "get_started": 0, "get_sum": 6589, "max": 3758096384, "put": 5, "put_sum": 6589, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:56.978 DEBUG:tasks.ceph_test_case:wait_until_true: success in 30s and 0 retries 2024-09-13T16:53:56.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:57.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.150+0000 7f0e5ce00700 1 Processor -- start 2024-09-13T16:53:57.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.150+0000 7f0e5ce00700 1 -- start start 2024-09-13T16:53:57.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.150+0000 7f0e5ce00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.150+0000 7f0e5ce00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0e5806ea90 0x7f0e580ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.150+0000 7f0e5ce00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5806dd60 con 0x7f0e5806eed0 2024-09-13T16:53:57.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.150+0000 7f0e5ce00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e580cc5a0 con 0x7f0e5806ea90 2024-09-13T16:53:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e5ce00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b3250 con 0x7f0e5815aca0 2024-09-13T16:53:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0e5806ea90 0x7f0e580ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56a00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0e5806ea90 0x7f0e580ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:45458/0 (socket says 172.21.15.3:45458) 2024-09-13T16:53:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56a00700 1 -- 172.21.15.3:0/283239781 learned_addr learned my addr 172.21.15.3:0/283239781 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e57400700 1 --2- 172.21.15.3:0/283239781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56a00700 1 -- 172.21.15.3:0/283239781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 msgr2=0x7f0e580cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:57.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56a00700 1 --2- 172.21.15.3:0/283239781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56a00700 1 -- 172.21.15.3:0/283239781 >> v1:172.21.15.157:6789/0 conn(0x7f0e5806eed0 legacy=0x7f0e580cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56a00700 1 -- 172.21.15.3:0/283239781 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580cc7b0 con 0x7f0e5806ea90 2024-09-13T16:53:57.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56000700 1 -- 172.21.15.3:0/283239781 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588249119 0 0) 0x7f0e5806dd60 con 0x7f0e5806eed0 2024-09-13T16:53:57.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56a00700 1 --2- 172.21.15.3:0/283239781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806ea90 0x7f0e580ce8b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009910 tx=0x7f0e44009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed538e545c002780 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56000700 1 -- 172.21.15.3:0/283239781 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44011830 con 0x7f0e5806ea90 2024-09-13T16:53:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56000700 1 -- 172.21.15.3:0/283239781 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e44011990 con 0x7f0e5806ea90 2024-09-13T16:53:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e56000700 1 -- 172.21.15.3:0/283239781 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44011d40 con 0x7f0e5806ea90 2024-09-13T16:53:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e5ce00700 1 -- 172.21.15.3:0/283239781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806ea90 msgr2=0x7f0e580ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/283239781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806ea90 0x7f0e580ce8b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009910 tx=0x7f0e44009c20 comp rx=0 tx=0).stop 2024-09-13T16:53:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e5ce00700 1 -- 172.21.15.3:0/283239781 shutdown_connections 2024-09-13T16:53:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/283239781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806ea90 0x7f0e580ce8b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/283239781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.154+0000 7f0e5ce00700 1 -- 172.21.15.3:0/283239781 >> 172.21.15.3:0/283239781 conn(0x7f0e580b24c0 msgr2=0x7f0e580b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.158+0000 7f0e5ce00700 1 -- 172.21.15.3:0/283239781 shutdown_connections 2024-09-13T16:53:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.158+0000 7f0e5ce00700 1 -- 172.21.15.3:0/283239781 wait complete. 2024-09-13T16:53:57.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.158+0000 7f0e5ce00700 1 Processor -- start 2024-09-13T16:53:57.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.158+0000 7f0e5ce00700 1 -- start start 2024-09-13T16:53:57.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e5ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5806ea90 0x7f0e58155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e5ce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806eed0 0x7f0e58153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e5ce00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e5ce00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b5ed0 con 0x7f0e5806ea90 2024-09-13T16:53:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e5ce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b61b0 con 0x7f0e5806eed0 2024-09-13T16:53:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e5ce00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b6490 con 0x7f0e5815aca0 2024-09-13T16:53:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5806ea90 0x7f0e58155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5806ea90 0x7f0e58155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40246/0 (socket says 172.21.15.3:40246) 2024-09-13T16:53:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 -- 172.21.15.3:0/3108104023 learned_addr learned my addr 172.21.15.3:0/3108104023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e56a00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806eed0 0x7f0e58153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57e00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 -- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 msgr2=0x7f0e580b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 -- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806eed0 msgr2=0x7f0e58153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806eed0 0x7f0e58153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 -- 172.21.15.3:0/3108104023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44009570 con 0x7f0e5806ea90 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57e00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e57400700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5806ea90 0x7f0e58155590 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e840 tx=0x7f0e4800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e4f400700 1 -- 172.21.15.3:0/3108104023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480095a0 con 0x7f0e5806ea90 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e4f400700 1 -- 172.21.15.3:0/3108104023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48013680 con 0x7f0e5806ea90 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5815c700 con 0x7f0e5806ea90 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e4f400700 1 -- 172.21.15.3:0/3108104023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480139f0 con 0x7f0e5806ea90 2024-09-13T16:53:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.162+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e580b5b90 con 0x7f0e5806ea90 2024-09-13T16:53:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.166+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5815c890 con 0x7f0e5806ea90 2024-09-13T16:53:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.166+0000 7f0e4f400700 1 -- 172.21.15.3:0/3108104023 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e48013b50 con 0x7f0e5806ea90 2024-09-13T16:53:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.166+0000 7f0e4f400700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0e38066730 0x7f0e38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.166+0000 7f0e4f400700 1 -- 172.21.15.3:0/3108104023 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f0e48084380 con 0x7f0e5806ea90 2024-09-13T16:53:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.170+0000 7f0e56a00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0e38066730 0x7f0e38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.170+0000 7f0e56a00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0e38066730 0x7f0e38068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009910 tx=0x7f0e44008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.174+0000 7f0e4f400700 1 -- 172.21.15.3:0/3108104023 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0e48051420 con 0x7f0e5806ea90 2024-09-13T16:53:57.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.406+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e580cc060 con 0x7f0e5806ea90 2024-09-13T16:53:57.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e4f400700 1 -- 172.21.15.3:0/3108104023 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f0e48054a40 con 0x7f0e5806ea90 2024-09-13T16:53:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:57.417 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0e38066730 msgr2=0x7f0e38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0e38066730 0x7f0e38068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009910 tx=0x7f0e44008030 comp rx=0 tx=0).stop 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5806ea90 msgr2=0x7f0e58155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5806ea90 0x7f0e58155590 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e840 tx=0x7f0e4800eb50 comp rx=0 tx=0).stop 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 shutdown_connections 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f0e38066730 0x7f0e38068bf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815aca0 0x7f0e580b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e5806eed0 0x7f0e58153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 --2- 172.21.15.3:0/3108104023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5806ea90 0x7f0e58155590 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 >> 172.21.15.3:0/3108104023 conn(0x7f0e580b24c0 msgr2=0x7f0e580af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 shutdown_connections 2024-09-13T16:53:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.410+0000 7f0e5ce00700 1 -- 172.21.15.3:0/3108104023 wait complete. 2024-09-13T16:53:57.422 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:57.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:57.643 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":82,"msgr_send_messages":37,"msgr_recv_bytes":36622,"msgr_send_bytes":31410,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.032649443,"msgr_running_send_time":0.010646829,"msgr_running_recv_time":0.011019879,"msgr_running_fast_dispatch_time":0.004759633,"msgr_send_messages_queue_lat":{"avgcount":36,"sum":0.010786817,"avgtime":0.000299633},"msgr_handle_ack_lat":{"avgcount":65,"sum":0.000018444,"avgtime":0.000000283},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":62,"msgr_send_messages":57,"msgr_recv_bytes":121592,"msgr_send_bytes":29412,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.016060060,"msgr_running_send_time":0.004050013,"msgr_running_recv_time":0.017084545,"msgr_running_fast_dispatch_time":0.001114479,"msgr_send_messages_queue_lat":{"avgcount":57,"sum":0.012310684,"avgtime":0.000215976},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":106608,"msgr_send_encrypted_bytes":12144},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":20,"msgr_recv_bytes":3670,"msgr_send_bytes":75764,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006826444,"msgr_running_send_time":0.001638751,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":19,"sum":0.006251870,"avgtime":0.000329045},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":73088},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":59,"sum":0.007982537,"avgtime":0.000135297}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":1,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":5,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4202044,"rdpos":4195196,"jlat":{"avgcount":5,"sum":0.018869506,"avgtime":0.003773901},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":2,"cap-":1,"rss":48600,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":3,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":991,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":254896,"buffer_anon_items":78,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56543,"mds_co_items":238,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":58,"op_send_bytes":8155,"op_resend":0,"op_reply":58,"op_latency":{"avgcount":58,"sum":0.171997174,"avgtime":0.002965468},"op_inflight":0,"oplen_avg":{"avgcount":58,"sum":116},"op":58,"op_r":36,"op_w":22,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":5,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":16,"omap_rd":48,"omap_del":5},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":3,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":148,"get_sum":145368,"get_or_fail_fail":0,"get_or_fail_success":148,"take":0,"take_sum":0,"put":148,"put_sum":145368,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":58,"take_sum":9042,"put":58,"put_sum":9042,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":58,"take_sum":58,"put":58,"put_sum":58,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":5,"get_sum":6589,"get_or_fail_fail":0,"get_or_fail_success":5,"take":0,"take_sum":0,"put":5,"put_sum":6589,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:57.644 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 65, "avgtime": 2.83e-07, "sum": 1.8444e-05 }, "msgr_recv_bytes": 36622, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 82, "msgr_running_fast_dispatch_time": 0.004759633, "msgr_running_recv_time": 0.011019879, "msgr_running_send_time": 0.010646829, "msgr_running_total_time": 0.032649443, "msgr_send_bytes": 31410, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 37, "msgr_send_messages_queue_lat": { "avgcount": 36, "avgtime": 0.000299633, "sum": 0.010786817 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 121592, "msgr_recv_encrypted_bytes": 106608, "msgr_recv_messages": 62, "msgr_running_fast_dispatch_time": 0.001114479, "msgr_running_recv_time": 0.017084545, "msgr_running_send_time": 0.004050013, "msgr_running_total_time": 0.01606006, "msgr_send_bytes": 29412, "msgr_send_encrypted_bytes": 12144, "msgr_send_messages": 57, "msgr_send_messages_queue_lat": { "avgcount": 57, "avgtime": 0.000215976, "sum": 0.012310684 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.001638751, "msgr_running_total_time": 0.006826444, "msgr_send_bytes": 75764, "msgr_send_encrypted_bytes": 73088, "msgr_send_messages": 20, "msgr_send_messages_queue_lat": { "avgcount": 19, "avgtime": 0.000329045, "sum": 0.00625187 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 59, "avgtime": 0.000135297, "sum": 0.007982537 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 5, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.003773901, "sum": 0.018869506 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4202044 }, "mds_mem": { "cap": 1, "cap+": 2, "cap-": 1, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 48600 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 3, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 991, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 254896, "buffer_anon_items": 78, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 56543, "mds_co_items": 238, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 5, "omap_rd": 48, "omap_wr": 16, "op": 58, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 58, "avgtime": 0.002965468, "sum": 0.171997174 }, "op_pg": 0, "op_r": 36, "op_reply": 58, "op_resend": 0, "op_rmw": 0, "op_send": 58, "op_send_bytes": 8155, "op_w": 22, "oplen_avg": { "avgcount": 58, "sum": 116 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 3 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 148, "get_or_fail_fail": 0, "get_or_fail_success": 148, "get_started": 0, "get_sum": 145368, "max": 104857600, "put": 148, "put_sum": 145368, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 58, "put_sum": 9042, "take": 58, "take_sum": 9042, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 58, "put_sum": 58, "take": 58, "take_sum": 58, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 5, "get_or_fail_fail": 0, "get_or_fail_success": 5, "get_started": 0, "get_sum": 6589, "max": 3758096384, "put": 5, "put_sum": 6589, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:57.644 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-13T16:53:57.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:57.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.806+0000 7f7f3fe00700 1 Processor -- start 2024-09-13T16:53:57.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.806+0000 7f7f3fe00700 1 -- start start 2024-09-13T16:53:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.806+0000 7f7f3fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4015cc40 0x7f7f4006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.806+0000 7f7f3fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f4006d540 0x7f7f400ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.806+0000 7f7f3fe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f400adc60 con 0x7f7f4015d020 2024-09-13T16:53:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.806+0000 7f7f3fe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4006da50 con 0x7f7f4006d540 2024-09-13T16:53:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.806+0000 7f7f3fe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4006dd30 con 0x7f7f4015cc40 2024-09-13T16:53:57.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3ea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4015cc40 0x7f7f4006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3f400700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7f4015d020 0x7f7f401609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:43802/0 (socket says 172.21.15.3:43802) 2024-09-13T16:53:57.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3f400700 1 -- 172.21.15.3:0/2162048913 learned_addr learned my addr 172.21.15.3:0/2162048913 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:53:57.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f36000700 1 --2- 172.21.15.3:0/2162048913 >> v2:172.21.15.157:3300/0 conn(0x7f7f4006d540 0x7f7f400ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3e000700 1 -- 172.21.15.3:0/2162048913 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745230520 0 0) 0x7f7f400adc60 con 0x7f7f4015d020 2024-09-13T16:53:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3e000700 1 -- 172.21.15.3:0/2162048913 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f240035d0 con 0x7f7f4015d020 2024-09-13T16:53:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3ea00700 1 -- 172.21.15.3:0/2162048913 >> v2:172.21.15.157:3300/0 conn(0x7f7f4006d540 msgr2=0x7f7f400ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3ea00700 1 --2- 172.21.15.3:0/2162048913 >> v2:172.21.15.157:3300/0 conn(0x7f7f4006d540 0x7f7f400ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3ea00700 1 -- 172.21.15.3:0/2162048913 >> v1:172.21.15.157:6789/0 conn(0x7f7f4015d020 legacy=0x7f7f401609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3ea00700 1 -- 172.21.15.3:0/2162048913 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f400ad610 con 0x7f7f4015cc40 2024-09-13T16:53:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f36000700 1 --2- 172.21.15.3:0/2162048913 >> v2:172.21.15.157:3300/0 conn(0x7f7f4006d540 0x7f7f400ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:57.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3ea00700 1 --2- 172.21.15.3:0/2162048913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4015cc40 0x7f7f4006fc20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009d90 tx=0x7f7f3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7fbb15f7a87ee04 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:57.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3e000700 1 -- 172.21.15.3:0/2162048913 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000e070 con 0x7f7f4015cc40 2024-09-13T16:53:57.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3e000700 1 -- 172.21.15.3:0/2162048913 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3000bb20 con 0x7f7f4015cc40 2024-09-13T16:53:57.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3e000700 1 -- 172.21.15.3:0/2162048913 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30012430 con 0x7f7f4015cc40 2024-09-13T16:53:57.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3fe00700 1 -- 172.21.15.3:0/2162048913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4015cc40 msgr2=0x7f7f4006fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/2162048913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4015cc40 0x7f7f4006fc20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009d90 tx=0x7f7f3000b330 comp rx=0 tx=0).stop 2024-09-13T16:53:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.810+0000 7f7f3fe00700 1 -- 172.21.15.3:0/2162048913 shutdown_connections 2024-09-13T16:53:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.814+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/2162048913 >> v2:172.21.15.157:3300/0 conn(0x7f7f4006d540 0x7f7f400ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.814+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/2162048913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4015cc40 0x7f7f4006fc20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.814+0000 7f7f3fe00700 1 -- 172.21.15.3:0/2162048913 >> 172.21.15.3:0/2162048913 conn(0x7f7f4006a2f0 msgr2=0x7f7f4006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.814+0000 7f7f3fe00700 1 -- 172.21.15.3:0/2162048913 shutdown_connections 2024-09-13T16:53:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.814+0000 7f7f3fe00700 1 -- 172.21.15.3:0/2162048913 wait complete. 2024-09-13T16:53:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.814+0000 7f7f3fe00700 1 Processor -- start 2024-09-13T16:53:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.814+0000 7f7f3fe00700 1 -- start start 2024-09-13T16:53:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4006d540 0x7f7f40148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f4015cc40 0x7f7f40158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4015d020 0x7f7f40159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40148860 con 0x7f7f4015cc40 2024-09-13T16:53:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40159dd0 con 0x7f7f4015d020 2024-09-13T16:53:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40159250 con 0x7f7f4006d540 2024-09-13T16:53:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f36000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f4015cc40 0x7f7f40158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3ea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4006d540 0x7f7f40148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3ea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4006d540 0x7f7f40148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:38758/0 (socket says 172.21.15.3:38758) 2024-09-13T16:53:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f36000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f4015cc40 0x7f7f40158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40258/0 (socket says 172.21.15.3:40258) 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3ea00700 1 -- 172.21.15.3:0/423328782 learned_addr learned my addr 172.21.15.3:0/423328782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3f400700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4015d020 0x7f7f40159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f36000700 1 -- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4006d540 msgr2=0x7f7f40148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f36000700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4006d540 0x7f7f40148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f36000700 1 -- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4015d020 msgr2=0x7f7f40159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f36000700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4015d020 0x7f7f40159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f36000700 1 -- 172.21.15.3:0/423328782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f280094b0 con 0x7f7f4015cc40 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3ea00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4006d540 0x7f7f40148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f36000700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f4015cc40 0x7f7f40158a00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9b0 tx=0x7f7f2800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:57.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f37400700 1 -- 172.21.15.3:0/423328782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800c850 con 0x7f7f4015cc40 2024-09-13T16:53:57.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f30009a70 con 0x7f7f4015cc40 2024-09-13T16:53:57.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f37400700 1 -- 172.21.15.3:0/423328782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2800c9b0 con 0x7f7f4015cc40 2024-09-13T16:53:57.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f37400700 1 -- 172.21.15.3:0/423328782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800cd40 con 0x7f7f4015cc40 2024-09-13T16:53:57.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f40159810 con 0x7f7f4015cc40 2024-09-13T16:53:57.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.818+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4006fc20 con 0x7f7f4015cc40 2024-09-13T16:53:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.822+0000 7f7f37400700 1 -- 172.21.15.3:0/423328782 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f28013470 con 0x7f7f4015cc40 2024-09-13T16:53:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.822+0000 7f7f37400700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7f1c066730 0x7f7f1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.822+0000 7f7f37400700 1 -- 172.21.15.3:0/423328782 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f7f28084790 con 0x7f7f4015cc40 2024-09-13T16:53:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.826+0000 7f7f3ea00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7f1c066730 0x7f7f1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:57.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.826+0000 7f7f3ea00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7f1c066730 0x7f7f1c068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7f30000f80 tx=0x7f7f30011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:57.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:57.830+0000 7f7f37400700 1 -- 172.21.15.3:0/423328782 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7f28051840 con 0x7f7f4015cc40 2024-09-13T16:53:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.054+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f40157420 con 0x7f7f4015cc40 2024-09-13T16:53:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f37400700 1 -- 172.21.15.3:0/423328782 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f7f2800cea0 con 0x7f7f4015cc40 2024-09-13T16:53:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:58.065 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:53:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7f1c066730 msgr2=0x7f7f1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7f1c066730 0x7f7f1c068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7f30000f80 tx=0x7f7f30011000 comp rx=0 tx=0).stop 2024-09-13T16:53:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f4015cc40 msgr2=0x7f7f40158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f4015cc40 0x7f7f40158a00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9b0 tx=0x7f7f2800ecc0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 shutdown_connections 2024-09-13T16:53:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7f1c066730 0x7f7f1c068bf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4015d020 0x7f7f40159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f4015cc40 0x7f7f40158a00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 --2- 172.21.15.3:0/423328782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f4006d540 0x7f7f40148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.058+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 >> 172.21.15.3:0/423328782 conn(0x7f7f4006a2f0 msgr2=0x7f7f400704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.062+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 shutdown_connections 2024-09-13T16:53:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.062+0000 7f7f3fe00700 1 -- 172.21.15.3:0/423328782 wait complete. 2024-09-13T16:53:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:53:58.090 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T16:53:58.269 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":82,"msgr_send_messages":37,"msgr_recv_bytes":36622,"msgr_send_bytes":31410,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.032649443,"msgr_running_send_time":0.010646829,"msgr_running_recv_time":0.011019879,"msgr_running_fast_dispatch_time":0.004759633,"msgr_send_messages_queue_lat":{"avgcount":36,"sum":0.010786817,"avgtime":0.000299633},"msgr_handle_ack_lat":{"avgcount":65,"sum":0.000018444,"avgtime":0.000000283},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":62,"msgr_send_messages":57,"msgr_recv_bytes":121592,"msgr_send_bytes":29412,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.016060060,"msgr_running_send_time":0.004050013,"msgr_running_recv_time":0.017084545,"msgr_running_fast_dispatch_time":0.001114479,"msgr_send_messages_queue_lat":{"avgcount":57,"sum":0.012310684,"avgtime":0.000215976},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":106608,"msgr_send_encrypted_bytes":12144},"AsyncMessenger::Worker-2":{"msgr_recv_messages":8,"msgr_send_messages":20,"msgr_recv_bytes":3670,"msgr_send_bytes":75764,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006826444,"msgr_running_send_time":0.001638751,"msgr_running_recv_time":0.004311050,"msgr_running_fast_dispatch_time":0.000262665,"msgr_send_messages_queue_lat":{"avgcount":19,"sum":0.006251870,"avgtime":0.000329045},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":73088},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":59,"sum":0.007982537,"avgtime":0.000135297}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000150212,"avgtime":0.000037553}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005727940,"avgtime":0.002863970},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":1,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":5,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4202044,"rdpos":4195196,"jlat":{"avgcount":5,"sum":0.018869506,"avgtime":0.003773901},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":2,"cap-":1,"rss":48600,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":3,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005375664,"avgtime":0.005375664},"req_getattr_latency":{"avgcount":1,"sum":0.000352276,"avgtime":0.000352276},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":991,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":254896,"buffer_anon_items":78,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5976,"osdmap_items":97,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56543,"mds_co_items":238,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":58,"op_send_bytes":8155,"op_resend":0,"op_reply":58,"op_latency":{"avgcount":58,"sum":0.171997174,"avgtime":0.002965468},"op_inflight":0,"oplen_avg":{"avgcount":58,"sum":116},"op":58,"op_r":36,"op_w":22,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":5,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":64,"map_full":0,"map_inc":2,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":16,"omap_rd":48,"omap_del":5},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":3,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":148,"get_sum":145368,"get_or_fail_fail":0,"get_or_fail_success":148,"take":0,"take_sum":0,"put":148,"put_sum":145368,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":58,"take_sum":9042,"put":58,"put_sum":9042,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":58,"take_sum":58,"put":58,"put_sum":58,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ee0ff490e0":{"val":0,"max":3758096384,"get_started":0,"get":5,"get_sum":6589,"get_or_fail_fail":0,"get_or_fail_success":5,"take":0,"take_sum":0,"put":5,"put_sum":6589,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T16:53:58.271 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 65, "avgtime": 2.83e-07, "sum": 1.8444e-05 }, "msgr_recv_bytes": 36622, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 82, "msgr_running_fast_dispatch_time": 0.004759633, "msgr_running_recv_time": 0.011019879, "msgr_running_send_time": 0.010646829, "msgr_running_total_time": 0.032649443, "msgr_send_bytes": 31410, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 37, "msgr_send_messages_queue_lat": { "avgcount": 36, "avgtime": 0.000299633, "sum": 0.010786817 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 121592, "msgr_recv_encrypted_bytes": 106608, "msgr_recv_messages": 62, "msgr_running_fast_dispatch_time": 0.001114479, "msgr_running_recv_time": 0.017084545, "msgr_running_send_time": 0.004050013, "msgr_running_total_time": 0.01606006, "msgr_send_bytes": 29412, "msgr_send_encrypted_bytes": 12144, "msgr_send_messages": 57, "msgr_send_messages_queue_lat": { "avgcount": 57, "avgtime": 0.000215976, "sum": 0.012310684 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3670, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 8, "msgr_running_fast_dispatch_time": 0.000262665, "msgr_running_recv_time": 0.00431105, "msgr_running_send_time": 0.001638751, "msgr_running_total_time": 0.006826444, "msgr_send_bytes": 75764, "msgr_send_encrypted_bytes": 73088, "msgr_send_messages": 20, "msgr_send_messages_queue_lat": { "avgcount": 19, "avgtime": 0.000329045, "sum": 0.00625187 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 59, "avgtime": 0.000135297, "sum": 0.007982537 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 3.7553e-05, "sum": 0.000150212 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00286397, "sum": 0.00572794 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 5, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.003773901, "sum": 0.018869506 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4202044 }, "mds_mem": { "cap": 1, "cap+": 2, "cap-": 1, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 48600 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 3, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005375664, "sum": 0.005375664 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000352276, "sum": 0.000352276 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 991, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 254896, "buffer_anon_items": 78, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 56543, "mds_co_items": 238, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5976, "osdmap_items": 97, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 64, "map_full": 0, "map_inc": 2, "omap_del": 5, "omap_rd": 48, "omap_wr": 16, "op": 58, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 58, "avgtime": 0.002965468, "sum": 0.171997174 }, "op_pg": 0, "op_r": 36, "op_reply": 58, "op_resend": 0, "op_rmw": 0, "op_send": 58, "op_send_bytes": 8155, "op_w": 22, "oplen_avg": { "avgcount": 58, "sum": 116 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 3 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 148, "get_or_fail_fail": 0, "get_or_fail_success": 148, "get_started": 0, "get_sum": 145368, "max": 104857600, "put": 148, "put_sum": 145368, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 58, "put_sum": 9042, "take": 58, "take_sum": 9042, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 58, "put_sum": 58, "take": 58, "take_sum": 58, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ee0ff490e0": { "get": 5, "get_or_fail_fail": 0, "get_or_fail_success": 5, "get_started": 0, "get_sum": 6589, "max": 3758096384, "put": 5, "put_sum": 6589, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T16:53:58.271 INFO:tasks.cephfs.test_openfiletable:omap_total_removes:1 2024-09-13T16:53:58.271 INFO:tasks.cephfs.test_openfiletable:omap_total_kv_pairs:1 2024-09-13T16:53:58.271 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T16:53:58.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T16:53:58.313 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T16:53:58.361 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T16:53:58.361 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:53:58.361 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:53:58.378 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:53:58.378 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:53:58.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:53:58.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.666+0000 7f6b28e00700 1 Processor -- start 2024-09-13T16:53:58.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.666+0000 7f6b28e00700 1 -- start start 2024-09-13T16:53:58.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b28e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b2415cea0 0x7f6b240cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b28e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b240d3740 0x7f6b240cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b28e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b240c6ee0 0x7f6b240c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b28e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2415d280 con 0x7f6b240d3740 2024-09-13T16:53:58.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b28e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b240d14e0 con 0x7f6b2415cea0 2024-09-13T16:53:58.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b28e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240d17c0 con 0x7f6b240c6ee0 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b240d3740 0x7f6b240cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b240d3740 0x7f6b240cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40264/0 (socket says 172.21.15.3:40264) 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 -- 172.21.15.3:0/3768574564 learned_addr learned my addr 172.21.15.3:0/3768574564 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b23e00700 1 --2- 172.21.15.3:0/3768574564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b240c6ee0 0x7f6b240c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b23400700 1 --2- 172.21.15.3:0/3768574564 >> v2:172.21.15.157:3300/0 conn(0x7f6b2415cea0 0x7f6b240cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 -- 172.21.15.3:0/3768574564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b240c6ee0 msgr2=0x7f6b240c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 --2- 172.21.15.3:0/3768574564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b240c6ee0 0x7f6b240c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 -- 172.21.15.3:0/3768574564 >> v2:172.21.15.157:3300/0 conn(0x7f6b2415cea0 msgr2=0x7f6b240cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 --2- 172.21.15.3:0/3768574564 >> v2:172.21.15.157:3300/0 conn(0x7f6b2415cea0 0x7f6b240cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 -- 172.21.15.3:0/3768574564 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b240d1aa0 con 0x7f6b240d3740 2024-09-13T16:53:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22a00700 1 --2- 172.21.15.3:0/3768574564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240d3740 0x7f6b240cfa10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6b240a0a50 tx=0x7f6b10009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce828d3c80062d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22000700 1 -- 172.21.15.3:0/3768574564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000b8a0 con 0x7f6b240d3740 2024-09-13T16:53:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.670+0000 7f6b22000700 1 -- 172.21.15.3:0/3768574564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1000ba00 con 0x7f6b240d3740 2024-09-13T16:53:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b22000700 1 -- 172.21.15.3:0/3768574564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000bd90 con 0x7f6b240d3740 2024-09-13T16:53:58.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 -- 172.21.15.3:0/3768574564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240d3740 msgr2=0x7f6b240cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3768574564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240d3740 0x7f6b240cfa10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6b240a0a50 tx=0x7f6b10009bf0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 -- 172.21.15.3:0/3768574564 shutdown_connections 2024-09-13T16:53:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3768574564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240d3740 0x7f6b240cfa10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3768574564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b240c6ee0 0x7f6b240c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3768574564 >> v2:172.21.15.157:3300/0 conn(0x7f6b2415cea0 0x7f6b240cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 -- 172.21.15.3:0/3768574564 >> 172.21.15.3:0/3768574564 conn(0x7f6b240d2b90 msgr2=0x7f6b240d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 -- 172.21.15.3:0/3768574564 shutdown_connections 2024-09-13T16:53:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.674+0000 7f6b28e00700 1 -- 172.21.15.3:0/3768574564 wait complete. 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b28e00700 1 Processor -- start 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b28e00700 1 -- start start 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b28e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240c6ee0 0x7f6b2414eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b28e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b240d3740 0x7f6b2409c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b28e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2415cea0 0x7f6b2409cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b28e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2415f2f0 con 0x7f6b240c6ee0 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b28e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2415f5d0 con 0x7f6b240d3740 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b28e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2415f8b0 con 0x7f6b2415cea0 2024-09-13T16:53:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240c6ee0 0x7f6b2414eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240c6ee0 0x7f6b2414eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40266/0 (socket says 172.21.15.3:40266) 2024-09-13T16:53:58.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23400700 1 -- 172.21.15.3:0/3489243424 learned_addr learned my addr 172.21.15.3:0/3489243424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:58.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23e00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2415cea0 0x7f6b2409cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b22a00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b240d3740 0x7f6b2409c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23400700 1 -- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2415cea0 msgr2=0x7f6b2409cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23400700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2415cea0 0x7f6b2409cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23400700 1 -- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b240d3740 msgr2=0x7f6b2409c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23400700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b240d3740 0x7f6b2409c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.678+0000 7f6b23400700 1 -- 172.21.15.3:0/3489243424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10009570 con 0x7f6b240c6ee0 2024-09-13T16:53:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.682+0000 7f6b23400700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240c6ee0 0x7f6b2414eec0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d800 tx=0x7f6b1400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.682+0000 7f6b1b400700 1 -- 172.21.15.3:0/3489243424 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140125f0 con 0x7f6b240c6ee0 2024-09-13T16:53:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.682+0000 7f6b1b400700 1 -- 172.21.15.3:0/3489243424 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b14012750 con 0x7f6b240c6ee0 2024-09-13T16:53:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.682+0000 7f6b1b400700 1 -- 172.21.15.3:0/3489243424 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14012ac0 con 0x7f6b240c6ee0 2024-09-13T16:53:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.682+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2415fb90 con 0x7f6b240c6ee0 2024-09-13T16:53:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.682+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2414f400 con 0x7f6b240c6ee0 2024-09-13T16:53:58.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.682+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2409e800 con 0x7f6b240c6ee0 2024-09-13T16:53:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.686+0000 7f6b1b400700 1 -- 172.21.15.3:0/3489243424 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b14012c20 con 0x7f6b240c6ee0 2024-09-13T16:53:58.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.686+0000 7f6b1b400700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b04066730 0x7f6b04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.686+0000 7f6b1b400700 1 -- 172.21.15.3:0/3489243424 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f6b14084470 con 0x7f6b240c6ee0 2024-09-13T16:53:58.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.686+0000 7f6b22a00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b04066730 0x7f6b04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.686+0000 7f6b22a00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b04066730 0x7f6b04068bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009860 tx=0x7f6b10008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:58.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.690+0000 7f6b1b400700 1 -- 172.21.15.3:0/3489243424 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b14051510 con 0x7f6b240c6ee0 2024-09-13T16:53:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.878+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b240c7370 con 0x7f6b240c6ee0 2024-09-13T16:53:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.882+0000 7f6b1b400700 1 -- 172.21.15.3:0/3489243424 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17078 (secure 0 0 0) 0x7f6b14054b30 con 0x7f6b240c6ee0 2024-09-13T16:53:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:53:58.890 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":64,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:53:15.879157+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:58.182202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:59.098263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6834/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.157:6836/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.3:6835/291442393":"2024-09-14T16:53:08.782245+0000","172.21.15.3:6834/291442393":"2024-09-14T16:53:08.782245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b04066730 msgr2=0x7f6b04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b04066730 0x7f6b04068bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009860 tx=0x7f6b10008030 comp rx=0 tx=0).stop 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240c6ee0 msgr2=0x7f6b2414eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240c6ee0 0x7f6b2414eec0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d800 tx=0x7f6b1400db10 comp rx=0 tx=0).stop 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 shutdown_connections 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6b04066730 0x7f6b04068bf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2415cea0 0x7f6b2409cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b240d3740 0x7f6b2409c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 --2- 172.21.15.3:0/3489243424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b240c6ee0 0x7f6b2414eec0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 >> 172.21.15.3:0/3489243424 conn(0x7f6b240d2b90 msgr2=0x7f6b240c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 shutdown_connections 2024-09-13T16:53:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:58.886+0000 7f6b28e00700 1 -- 172.21.15.3:0/3489243424 wait complete. 2024-09-13T16:53:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:52:58.182202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:52:59.098263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:53:58.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:53:58.973 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:53:58.978 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:53:59.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.098+0000 7ff9fcc00700 1 Processor -- start 2024-09-13T16:53:59.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.098+0000 7ff9fcc00700 1 -- start start 2024-09-13T16:53:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9fcc00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9f815e470 0x7ff9f80b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9fcc00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9f80cf840 0x7ff9f80b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9fcc00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80d1ae0 con 0x7ff9f80d1670 2024-09-13T16:53:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9fcc00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f80b21d0 con 0x7ff9f80cf840 2024-09-13T16:53:59.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9fcc00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f80cb810 con 0x7ff9f815e470 2024-09-13T16:53:59.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f6800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9f80cf840 0x7ff9f80b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f6800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9f80cf840 0x7ff9f80b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40278/0 (socket says 172.21.15.3:40278) 2024-09-13T16:53:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f6800700 1 -- 172.21.15.3:0/3307680428 learned_addr learned my addr 172.21.15.3:0/3307680428 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f5e00700 1 --2- 172.21.15.3:0/3307680428 >> v2:172.21.15.157:3300/0 conn(0x7ff9f815e470 0x7ff9f80b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f6800700 1 -- 172.21.15.3:0/3307680428 >> v2:172.21.15.157:3300/0 conn(0x7ff9f815e470 msgr2=0x7ff9f80b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f6800700 1 --2- 172.21.15.3:0/3307680428 >> v2:172.21.15.157:3300/0 conn(0x7ff9f815e470 0x7ff9f80b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f6800700 1 -- 172.21.15.3:0/3307680428 >> v1:172.21.15.157:6789/0 conn(0x7ff9f80d1670 legacy=0x7ff9f80cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f6800700 1 -- 172.21.15.3:0/3307680428 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80cba90 con 0x7ff9f80cf840 2024-09-13T16:53:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f6800700 1 --2- 172.21.15.3:0/3307680428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b03b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80d0b20 tx=0x7ff9f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3054172b03c8ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f4a00700 1 -- 172.21.15.3:0/3307680428 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000e070 con 0x7ff9f80cf840 2024-09-13T16:53:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f4a00700 1 -- 172.21.15.3:0/3307680428 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f000b910 con 0x7ff9f80cf840 2024-09-13T16:53:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9f4a00700 1 -- 172.21.15.3:0/3307680428 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000bcc0 con 0x7ff9f80cf840 2024-09-13T16:53:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9fcc00700 1 -- 172.21.15.3:0/3307680428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 msgr2=0x7ff9f80b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.102+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/3307680428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b03b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80d0b20 tx=0x7ff9f000b330 comp rx=0 tx=0).stop 2024-09-13T16:53:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.106+0000 7ff9fcc00700 1 -- 172.21.15.3:0/3307680428 shutdown_connections 2024-09-13T16:53:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.106+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/3307680428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b03b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.106+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/3307680428 >> v2:172.21.15.157:3300/0 conn(0x7ff9f815e470 0x7ff9f80b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.106+0000 7ff9fcc00700 1 -- 172.21.15.3:0/3307680428 >> 172.21.15.3:0/3307680428 conn(0x7ff9f80d0b50 msgr2=0x7ff9f80d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.106+0000 7ff9fcc00700 1 -- 172.21.15.3:0/3307680428 shutdown_connections 2024-09-13T16:53:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.106+0000 7ff9fcc00700 1 -- 172.21.15.3:0/3307680428 wait complete. 2024-09-13T16:53:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.106+0000 7ff9fcc00700 1 Processor -- start 2024-09-13T16:53:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9fcc00700 1 -- start start 2024-09-13T16:53:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9fcc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9fcc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f80d1670 0x7ff9f81f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9fcc00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9f815e470 0x7ff9f80c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9fcc00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80cab70 con 0x7ff9f80cf840 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9fcc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80b0320 con 0x7ff9f80d1670 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9fcc00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80ca890 con 0x7ff9f815e470 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40288/0 (socket says 172.21.15.3:40288) 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5e00700 1 -- 172.21.15.3:0/1019313102 learned_addr learned my addr 172.21.15.3:0/1019313102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f6800700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9f815e470 0x7ff9f80c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5400700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f80d1670 0x7ff9f81f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5e00700 1 -- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9f815e470 msgr2=0x7ff9f80c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5e00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9f815e470 0x7ff9f80c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5e00700 1 -- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f80d1670 msgr2=0x7ff9f81f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5e00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f80d1670 0x7ff9f81f3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.110+0000 7ff9f5e00700 1 -- 172.21.15.3:0/1019313102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f0009b30 con 0x7ff9f80cf840 2024-09-13T16:53:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.114+0000 7ff9f5e00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b41e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800ebe0 tx=0x7ff9e800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.114+0000 7ff9ee000700 1 -- 172.21.15.3:0/1019313102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8009740 con 0x7ff9f80cf840 2024-09-13T16:53:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.114+0000 7ff9ee000700 1 -- 172.21.15.3:0/1019313102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e800c850 con 0x7ff9f80cf840 2024-09-13T16:53:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.114+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f80afcd0 con 0x7ff9f80cf840 2024-09-13T16:53:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.114+0000 7ff9ee000700 1 -- 172.21.15.3:0/1019313102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800cbe0 con 0x7ff9f80cf840 2024-09-13T16:53:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.114+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f81f39c0 con 0x7ff9f80cf840 2024-09-13T16:53:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.114+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f815e850 con 0x7ff9f80cf840 2024-09-13T16:53:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.118+0000 7ff9ee000700 1 -- 172.21.15.3:0/1019313102 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9e800cd40 con 0x7ff9f80cf840 2024-09-13T16:53:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.118+0000 7ff9ee000700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff9d8066730 0x7ff9d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.118+0000 7ff9ee000700 1 -- 172.21.15.3:0/1019313102 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff9e8084a20 con 0x7ff9f80cf840 2024-09-13T16:53:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.118+0000 7ff9f5400700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff9d8066730 0x7ff9d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.122+0000 7ff9f5400700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff9d8066730 0x7ff9d8068bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80b3a30 tx=0x7ff9e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.122+0000 7ff9ee000700 1 -- 172.21.15.3:0/1019313102 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff9e8051cd0 con 0x7ff9f80cf840 2024-09-13T16:53:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.298+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9f80cb110 con 0x7ff9f80cf840 2024-09-13T16:53:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.298+0000 7ff9ee000700 1 -- 172.21.15.3:0/1019313102 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9e80552f0 con 0x7ff9f80cf840 2024-09-13T16:53:59.305 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:53:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff9d8066730 msgr2=0x7ff9d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff9d8066730 0x7ff9d8068bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80b3a30 tx=0x7ff9e4009040 comp rx=0 tx=0).stop 2024-09-13T16:53:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 msgr2=0x7ff9f80b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b41e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800ebe0 tx=0x7ff9e800eef0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 shutdown_connections 2024-09-13T16:53:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff9d8066730 0x7ff9d8068bf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9f815e470 0x7ff9f80c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f80d1670 0x7ff9f81f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 --2- 172.21.15.3:0/1019313102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9f80cf840 0x7ff9f80b41e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 >> 172.21.15.3:0/1019313102 conn(0x7ff9f80d0b50 msgr2=0x7ff9f80cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 shutdown_connections 2024-09-13T16:53:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.302+0000 7ff9fcc00700 1 -- 172.21.15.3:0/1019313102 wait complete. 2024-09-13T16:53:59.329 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:53:59.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.510+0000 7fe96f400700 1 Processor -- start 2024-09-13T16:53:59.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.510+0000 7fe96f400700 1 -- start start 2024-09-13T16:53:59.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.510+0000 7fe96f400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9680ce470 0x7fe9680b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.510+0000 7fe96f400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680b29d0 0x7fe9680ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.510+0000 7fe96f400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9680cd130 con 0x7fe968071e60 2024-09-13T16:53:59.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.510+0000 7fe96f400700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9680b2e60 con 0x7fe9680ce470 2024-09-13T16:53:59.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.510+0000 7fe96f400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680cd3d0 con 0x7fe9680b29d0 2024-09-13T16:53:59.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9680ce470 0x7fe9680b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9680ce470 0x7fe9680b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40292/0 (socket says 172.21.15.3:40292) 2024-09-13T16:53:59.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967e00700 1 -- 172.21.15.3:0/3083525857 learned_addr learned my addr 172.21.15.3:0/3083525857 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:59.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96d400700 1 --2- 172.21.15.3:0/3083525857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680b29d0 0x7fe9680ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967e00700 1 -- 172.21.15.3:0/3083525857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680b29d0 msgr2=0x7fe9680ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:59.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967e00700 1 --2- 172.21.15.3:0/3083525857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680b29d0 0x7fe9680ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967e00700 1 -- 172.21.15.3:0/3083525857 >> v1:172.21.15.157:6789/0 conn(0x7fe968071e60 legacy=0x7fe96815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967e00700 1 -- 172.21.15.3:0/3083525857 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680cd6b0 con 0x7fe9680ce470 2024-09-13T16:53:59.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967e00700 1 --2- 172.21.15.3:0/3083525857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680ce470 0x7fe9680b2490 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe954009910 tx=0x7fe954009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a7591ce6fba94c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967400700 1 -- 172.21.15.3:0/3083525857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400ba60 con 0x7fe9680ce470 2024-09-13T16:53:59.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967400700 1 -- 172.21.15.3:0/3083525857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95400bbc0 con 0x7fe9680ce470 2024-09-13T16:53:59.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe967400700 1 -- 172.21.15.3:0/3083525857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954009e60 con 0x7fe9680ce470 2024-09-13T16:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96f400700 1 -- 172.21.15.3:0/3083525857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680ce470 msgr2=0x7fe9680b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96f400700 1 --2- 172.21.15.3:0/3083525857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680ce470 0x7fe9680b2490 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe954009910 tx=0x7fe954009c20 comp rx=0 tx=0).stop 2024-09-13T16:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96f400700 1 -- 172.21.15.3:0/3083525857 shutdown_connections 2024-09-13T16:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96f400700 1 --2- 172.21.15.3:0/3083525857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680b29d0 0x7fe9680ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96f400700 1 --2- 172.21.15.3:0/3083525857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680ce470 0x7fe9680b2490 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96f400700 1 -- 172.21.15.3:0/3083525857 >> 172.21.15.3:0/3083525857 conn(0x7fe9680b07a0 msgr2=0x7fe9680b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96f400700 1 -- 172.21.15.3:0/3083525857 shutdown_connections 2024-09-13T16:53:59.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.514+0000 7fe96f400700 1 -- 172.21.15.3:0/3083525857 wait complete. 2024-09-13T16:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.518+0000 7fe96f400700 1 Processor -- start 2024-09-13T16:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.518+0000 7fe96f400700 1 -- start start 2024-09-13T16:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968071e60 0x7fe96814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680b29d0 0x7fe9681f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96f400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680ce470 0x7fe9681611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96f400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96814aaf0 con 0x7fe9680b29d0 2024-09-13T16:53:59.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96f400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96815ecd0 con 0x7fe968071e60 2024-09-13T16:53:59.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96f400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe968156e70 con 0x7fe9680ce470 2024-09-13T16:53:59.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680b29d0 0x7fe9681f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680b29d0 0x7fe9681f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40302/0 (socket says 172.21.15.3:40302) 2024-09-13T16:53:59.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 -- 172.21.15.3:0/2700132302 learned_addr learned my addr 172.21.15.3:0/2700132302 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:59.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96d400700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680ce470 0x7fe9681611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96ca00700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968071e60 0x7fe96814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 -- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680ce470 msgr2=0x7fe9681611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680ce470 0x7fe9681611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 -- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968071e60 msgr2=0x7fe96814a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968071e60 0x7fe96814a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 -- 172.21.15.3:0/2700132302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe954009570 con 0x7fe9680b29d0 2024-09-13T16:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96ca00700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968071e60 0x7fe96814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe967e00700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680b29d0 0x7fe9681f2b50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe954004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe964c00700 1 -- 172.21.15.3:0/2700132302 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954012a00 con 0x7fe9680b29d0 2024-09-13T16:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe964c00700 1 -- 172.21.15.3:0/2700132302 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95401e070 con 0x7fe9680b29d0 2024-09-13T16:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe964c00700 1 -- 172.21.15.3:0/2700132302 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954012da0 con 0x7fe9680b29d0 2024-09-13T16:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96815e0c0 con 0x7fe9680b29d0 2024-09-13T16:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.522+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96815e680 con 0x7fe9680b29d0 2024-09-13T16:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.526+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96815ee80 con 0x7fe9680b29d0 2024-09-13T16:53:59.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.526+0000 7fe964c00700 1 -- 172.21.15.3:0/2700132302 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe95402d450 con 0x7fe9680b29d0 2024-09-13T16:53:59.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.526+0000 7fe964c00700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe948066730 0x7fe948068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.530+0000 7fe964c00700 1 -- 172.21.15.3:0/2700132302 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fe954085460 con 0x7fe9680b29d0 2024-09-13T16:53:59.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.530+0000 7fe96ca00700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe948066730 0x7fe948068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.530+0000 7fe96ca00700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe948066730 0x7fe948068bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe95c000f60 tx=0x7fe95c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.534+0000 7fe964c00700 1 -- 172.21.15.3:0/2700132302 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe96815ee80 con 0x7fe9680b29d0 2024-09-13T16:53:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.726+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe96815ee80 con 0x7fe9680b29d0 2024-09-13T16:53:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.726+0000 7fe964c00700 1 -- 172.21.15.3:0/2700132302 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe96815ee80 con 0x7fe9680b29d0 2024-09-13T16:53:59.733 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:53:59.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe948066730 msgr2=0x7fe948068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe948066730 0x7fe948068bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe95c000f60 tx=0x7fe95c009040 comp rx=0 tx=0).stop 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680b29d0 msgr2=0x7fe9681f2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680b29d0 0x7fe9681f2b50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe954004770 comp rx=0 tx=0).stop 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 shutdown_connections 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe948066730 0x7fe948068bf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9680ce470 0x7fe9681611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9680b29d0 0x7fe9681f2b50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 --2- 172.21.15.3:0/2700132302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968071e60 0x7fe96814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 >> 172.21.15.3:0/2700132302 conn(0x7fe9680b07a0 msgr2=0x7fe96815a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 shutdown_connections 2024-09-13T16:53:59.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.730+0000 7fe96f400700 1 -- 172.21.15.3:0/2700132302 wait complete. 2024-09-13T16:53:59.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:53:59.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.926+0000 7f634b800700 1 Processor -- start 2024-09-13T16:53:59.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.926+0000 7f634b800700 1 -- start start 2024-09-13T16:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f634b800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f634415d0b0 0x7f63440b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f634b800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f634406c520 0x7f63440cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f634b800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63440d1ab0 con 0x7f63440cdf70 2024-09-13T16:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f634b800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63440cff60 con 0x7f634406c520 2024-09-13T16:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f634b800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63440d0200 con 0x7f634415d0b0 2024-09-13T16:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f6349800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f634406c520 0x7f63440cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f6349800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f634406c520 0x7f63440cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40304/0 (socket says 172.21.15.3:40304) 2024-09-13T16:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f6349800700 1 -- 172.21.15.3:0/2827811901 learned_addr learned my addr 172.21.15.3:0/2827811901 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f6348e00700 1 --2- 172.21.15.3:0/2827811901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f634415d0b0 0x7f63440b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f6343400700 1 -- 172.21.15.3:0/2827811901 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953849265 0 0) 0x7f63440d1ab0 con 0x7f63440cdf70 2024-09-13T16:53:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f6349800700 1 -- 172.21.15.3:0/2827811901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f634415d0b0 msgr2=0x7f63440b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.930+0000 7f6349800700 1 --2- 172.21.15.3:0/2827811901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f634415d0b0 0x7f63440b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f6349800700 1 -- 172.21.15.3:0/2827811901 >> v1:172.21.15.157:6789/0 conn(0x7f63440cdf70 legacy=0x7f634406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f6349800700 1 -- 172.21.15.3:0/2827811901 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63440d04e0 con 0x7f634406c520 2024-09-13T16:53:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f6349800700 1 --2- 172.21.15.3:0/2827811901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634406c520 0x7f63440cb540 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f633c009e50 tx=0x7f633c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fd43e342f3bfd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f6343400700 1 -- 172.21.15.3:0/2827811901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c00e070 con 0x7f634406c520 2024-09-13T16:53:59.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f6343400700 1 -- 172.21.15.3:0/2827811901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c00b910 con 0x7f634406c520 2024-09-13T16:53:59.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f6343400700 1 -- 172.21.15.3:0/2827811901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c00bcc0 con 0x7f634406c520 2024-09-13T16:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f634b800700 1 -- 172.21.15.3:0/2827811901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634406c520 msgr2=0x7f63440cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f634b800700 1 --2- 172.21.15.3:0/2827811901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634406c520 0x7f63440cb540 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f633c009e50 tx=0x7f633c00b330 comp rx=0 tx=0).stop 2024-09-13T16:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f634b800700 1 -- 172.21.15.3:0/2827811901 shutdown_connections 2024-09-13T16:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f634b800700 1 --2- 172.21.15.3:0/2827811901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634406c520 0x7f63440cb540 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f634b800700 1 --2- 172.21.15.3:0/2827811901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f634415d0b0 0x7f63440b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f634b800700 1 -- 172.21.15.3:0/2827811901 >> 172.21.15.3:0/2827811901 conn(0x7f63440d0b50 msgr2=0x7f63440d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f634b800700 1 -- 172.21.15.3:0/2827811901 shutdown_connections 2024-09-13T16:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.934+0000 7f634b800700 1 -- 172.21.15.3:0/2827811901 wait complete. 2024-09-13T16:53:59.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f634b800700 1 Processor -- start 2024-09-13T16:53:59.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f634b800700 1 -- start start 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f634b800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f634406c520 0x7f63440b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f634b800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63440cdf70 0x7f634414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f634b800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634415d0b0 0x7f63440c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f634b800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63440c6960 con 0x7f634415d0b0 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f634b800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634414f610 con 0x7f634406c520 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f634b800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63440c5d70 con 0x7f63440cdf70 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634415d0b0 0x7f63440c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634415d0b0 0x7f63440c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40306/0 (socket says 172.21.15.3:40306) 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 -- 172.21.15.3:0/1014405248 learned_addr learned my addr 172.21.15.3:0/1014405248 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:53:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6343e00700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63440cdf70 0x7f634414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6348e00700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f634406c520 0x7f63440b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 -- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63440cdf70 msgr2=0x7f634414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63440cdf70 0x7f634414f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 -- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f634406c520 msgr2=0x7f63440b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f634406c520 0x7f63440b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 -- 172.21.15.3:0/1014405248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c009b30 con 0x7f634415d0b0 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6348e00700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f634406c520 0x7f63440b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6343e00700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63440cdf70 0x7f634414f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6349800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634415d0b0 0x7f63440c6df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f633c012000 tx=0x7f633c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.938+0000 7f6340c00700 1 -- 172.21.15.3:0/1014405248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c0099b0 con 0x7f634415d0b0 2024-09-13T16:53:59.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.942+0000 7f6340c00700 1 -- 172.21.15.3:0/1014405248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c0092e0 con 0x7f634415d0b0 2024-09-13T16:53:59.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.942+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63440c6030 con 0x7f634415d0b0 2024-09-13T16:53:59.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.942+0000 7f6340c00700 1 -- 172.21.15.3:0/1014405248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c01c660 con 0x7f634415d0b0 2024-09-13T16:53:59.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.942+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63440c65f0 con 0x7f634415d0b0 2024-09-13T16:53:59.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.942+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63440b1bd0 con 0x7f634415d0b0 2024-09-13T16:53:59.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.942+0000 7f6340c00700 1 -- 172.21.15.3:0/1014405248 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f633c01c7c0 con 0x7f634415d0b0 2024-09-13T16:53:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.946+0000 7f6340c00700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6324066730 0x7f6324068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:59.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.946+0000 7f6340c00700 1 -- 172.21.15.3:0/1014405248 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f633c0860b0 con 0x7f634415d0b0 2024-09-13T16:53:59.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.946+0000 7f6348e00700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6324066730 0x7f6324068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:59.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.946+0000 7f6348e00700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6324066730 0x7f6324068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6338009ba0 tx=0x7f6338008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:53:59.950+0000 7f6340c00700 1 -- 172.21.15.3:0/1014405248 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f633c0531e0 con 0x7f634415d0b0 2024-09-13T16:54:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.134+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63440c6aa0 con 0x7f634415d0b0 2024-09-13T16:54:00.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f6340c00700 1 -- 172.21.15.3:0/1014405248 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7f633c056800 con 0x7f634415d0b0 2024-09-13T16:54:00.144 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6324066730 msgr2=0x7f6324068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6324066730 0x7f6324068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6338009ba0 tx=0x7f6338008040 comp rx=0 tx=0).stop 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634415d0b0 msgr2=0x7f63440c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634415d0b0 0x7f63440c6df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f633c012000 tx=0x7f633c004af0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 shutdown_connections 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6324066730 0x7f6324068bf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634415d0b0 0x7f63440c6df0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63440cdf70 0x7f634414f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 --2- 172.21.15.3:0/1014405248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f634406c520 0x7f63440b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 >> 172.21.15.3:0/1014405248 conn(0x7f63440d0b50 msgr2=0x7f634406be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:00.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 shutdown_connections 2024-09-13T16:54:00.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.138+0000 7f634b800700 1 -- 172.21.15.3:0/1014405248 wait complete. 2024-09-13T16:54:00.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:54:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.338+0000 7f38a6c00700 1 Processor -- start 2024-09-13T16:54:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.338+0000 7f38a6c00700 1 -- start start 2024-09-13T16:54:00.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a6c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38a0160f30 0x7f38a0165770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a6c00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a006ba80 con 0x7f38a01652e0 2024-09-13T16:54:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a6c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a006fad0 con 0x7f38a006d640 2024-09-13T16:54:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a6c00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a006bcd0 con 0x7f38a0160f30 2024-09-13T16:54:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a5800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38a0160f30 0x7f38a0165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a6200700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f38a01652e0 0x7f38a0070240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42590/0 (socket says 172.21.15.3:42590) 2024-09-13T16:54:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a6200700 1 -- 172.21.15.3:0/3066751480 learned_addr learned my addr 172.21.15.3:0/3066751480 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a5800700 1 -- 172.21.15.3:0/3066751480 >> v1:172.21.15.157:6789/0 conn(0x7f38a006d640 legacy=0x7f38a0164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a5800700 1 -- 172.21.15.3:0/3066751480 >> v1:172.21.15.3:6789/0 conn(0x7f38a01652e0 legacy=0x7f38a0070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a5800700 1 -- 172.21.15.3:0/3066751480 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a006bfb0 con 0x7f38a0160f30 2024-09-13T16:54:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.342+0000 7f38a5800700 1 --2- 172.21.15.3:0/3066751480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0165770 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3894009850 tx=0x7f3894009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54809d203dcb7f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f389fe00700 1 -- 172.21.15.3:0/3066751480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011830 con 0x7f38a0160f30 2024-09-13T16:54:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f389fe00700 1 -- 172.21.15.3:0/3066751480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3894011990 con 0x7f38a0160f30 2024-09-13T16:54:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f389fe00700 1 -- 172.21.15.3:0/3066751480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011d20 con 0x7f38a0160f30 2024-09-13T16:54:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 -- 172.21.15.3:0/3066751480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 msgr2=0x7f38a0165770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 --2- 172.21.15.3:0/3066751480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0165770 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3894009850 tx=0x7f3894009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 -- 172.21.15.3:0/3066751480 shutdown_connections 2024-09-13T16:54:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 --2- 172.21.15.3:0/3066751480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0165770 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 -- 172.21.15.3:0/3066751480 >> 172.21.15.3:0/3066751480 conn(0x7f38a006a2f0 msgr2=0x7f38a006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 -- 172.21.15.3:0/3066751480 shutdown_connections 2024-09-13T16:54:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 -- 172.21.15.3:0/3066751480 wait complete. 2024-09-13T16:54:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 Processor -- start 2024-09-13T16:54:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.346+0000 7f38a6c00700 1 -- start start 2024-09-13T16:54:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a006d640 0x7f38a0144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a01652e0 0x7f38a00a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a01eb4e0 con 0x7f38a0160f30 2024-09-13T16:54:00.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00a9ce0 con 0x7f38a006d640 2024-09-13T16:54:00.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a01eab40 con 0x7f38a01652e0 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0143830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40330/0 (socket says 172.21.15.3:40330) 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 -- 172.21.15.3:0/2298303868 learned_addr learned my addr 172.21.15.3:0/2298303868 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a5800700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a006d640 0x7f38a0144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6200700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a01652e0 0x7f38a00a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 -- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a01652e0 msgr2=0x7f38a00a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a01652e0 0x7f38a00a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 -- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a006d640 msgr2=0x7f38a0144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a006d640 0x7f38a0144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 -- 172.21.15.3:0/2298303868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940094b0 con 0x7f38a0160f30 2024-09-13T16:54:00.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a4e00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0143830 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c500 tx=0x7f388c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:00.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f389d600700 1 -- 172.21.15.3:0/2298303868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0052d0 con 0x7f38a0160f30 2024-09-13T16:54:00.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f389d600700 1 -- 172.21.15.3:0/2298303868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c005430 con 0x7f38a0160f30 2024-09-13T16:54:00.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a01eae20 con 0x7f38a0160f30 2024-09-13T16:54:00.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f389d600700 1 -- 172.21.15.3:0/2298303868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0055a0 con 0x7f38a0160f30 2024-09-13T16:54:00.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.350+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a0143d70 con 0x7f38a0160f30 2024-09-13T16:54:00.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.354+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a01eb620 con 0x7f38a0160f30 2024-09-13T16:54:00.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.354+0000 7f389d600700 1 -- 172.21.15.3:0/2298303868 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f388c0038e0 con 0x7f38a0160f30 2024-09-13T16:54:00.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.358+0000 7f389d600700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3880066730 0x7f3880068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.358+0000 7f389d600700 1 -- 172.21.15.3:0/2298303868 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f388c0836f0 con 0x7f38a0160f30 2024-09-13T16:54:00.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.358+0000 7f38a5800700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3880066730 0x7f3880068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.358+0000 7f38a5800700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3880066730 0x7f3880068bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3894009fd0 tx=0x7f3894008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:00.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.362+0000 7f389d600700 1 -- 172.21.15.3:0/2298303868 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f38a01eb620 con 0x7f38a0160f30 2024-09-13T16:54:00.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.554+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38a01eb620 con 0x7f38a0160f30 2024-09-13T16:54:00.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.554+0000 7f389d600700 1 -- 172.21.15.3:0/2298303868 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v64) v1 ==== 80+0+242 (secure 0 0 0) 0x7f38a01eb620 con 0x7f38a0160f30 2024-09-13T16:54:00.562 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.157:6834/2335957931 2024-09-14T16:53:08.803287+0000 2024-09-13T16:54:00.563 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.157:6836/2335957931 2024-09-14T16:53:08.803287+0000 2024-09-13T16:54:00.563 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6835/291442393 2024-09-14T16:53:08.782245+0000 2024-09-13T16:54:00.563 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6834/291442393 2024-09-14T16:53:08.782245+0000 2024-09-13T16:54:00.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3880066730 msgr2=0x7f3880068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3880066730 0x7f3880068bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3894009fd0 tx=0x7f3894008030 comp rx=0 tx=0).stop 2024-09-13T16:54:00.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 msgr2=0x7f38a0143830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0143830 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c500 tx=0x7f388c00c810 comp rx=0 tx=0).stop 2024-09-13T16:54:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 shutdown_connections 2024-09-13T16:54:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3880066730 0x7f3880068bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a01652e0 0x7f38a00a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a0160f30 0x7f38a0143830 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 --2- 172.21.15.3:0/2298303868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a006d640 0x7f38a0144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 >> 172.21.15.3:0/2298303868 conn(0x7f38a006a2f0 msgr2=0x7f38a00b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 shutdown_connections 2024-09-13T16:54:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.558+0000 7f38a6c00700 1 -- 172.21.15.3:0/2298303868 wait complete. 2024-09-13T16:54:00.567 INFO:teuthology.orchestra.run.smithi003.stderr:listed 4 entries 2024-09-13T16:54:00.590 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T16:54:00.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:00.591 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T16:54:00.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:54:00.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:00.633 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T16:54:00.698 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:54:00.698 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:54:00.718 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346844k,nr_inodes=4086711,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T16:54:00.718 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:54:00.718 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:54:00.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349932k,nr_inodes=4087483,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4619 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T16:54:00.756 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:54:00.756 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:54:00.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346844k,nr_inodes=4086711,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T16:54:00.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.970+0000 7fe24e000700 1 Processor -- start 2024-09-13T16:54:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.970+0000 7fe24e000700 1 -- start start 2024-09-13T16:54:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe24e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 0x7fe2480b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe24e000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe248071470 0x7fe24806d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe24e000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2480b3f10 con 0x7fe248142c20 2024-09-13T16:54:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe24e000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2480b3c30 con 0x7fe248071470 2024-09-13T16:54:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe24e000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24806cd30 con 0x7fe248142840 2024-09-13T16:54:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe248142c20 0x7fe248070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42606/0 (socket says 172.21.15.3:42606) 2024-09-13T16:54:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246a00700 1 -- 172.21.15.3:0/436760225 learned_addr learned my addr 172.21.15.3:0/436760225 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe247e00700 1 --2- 172.21.15.3:0/436760225 >> v2:172.21.15.3:3300/0 conn(0x7fe248071470 0x7fe24806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe247400700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 0x7fe2480b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861819071 0 0) 0x7fe2480b3f10 con 0x7fe248142c20 2024-09-13T16:54:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c0035d0 con 0x7fe248142c20 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773876859 0 0) 0x7fe22c0035d0 con 0x7fe248142c20 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 msgr2=0x7fe2480b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 0x7fe2480b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 >> v2:172.21.15.3:3300/0 conn(0x7fe248071470 msgr2=0x7fe24806d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 --2- 172.21.15.3:0/436760225 >> v2:172.21.15.3:3300/0 conn(0x7fe248071470 0x7fe24806d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24806d010 con 0x7fe248142c20 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fe2340017f0 con 0x7fe248142c20 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 >> v1:172.21.15.3:6789/0 conn(0x7fe248142c20 legacy=0x7fe248070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003a20 0x7fe22c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c004870 0x7fe22c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 0x7fe22c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480b3c30 con 0x7fe22c003a20 2024-09-13T16:54:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24806cd30 con 0x7fe22c004870 2024-09-13T16:54:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00dc40 con 0x7fe248142840 2024-09-13T16:54:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe247e00700 1 --2- 172.21.15.3:0/436760225 >> v2:172.21.15.3:3300/0 conn(0x7fe248071470 0x7fe24806d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe247400700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003a20 0x7fe22c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.974+0000 7fe246a00700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c004870 0x7fe22c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247e00700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 0x7fe22c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247400700 1 -- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 msgr2=0x7fe22c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247400700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 0x7fe22c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247400700 1 -- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c004870 msgr2=0x7fe22c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247400700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c004870 0x7fe22c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247400700 1 -- 172.21.15.3:0/436760225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0035d0 con 0x7fe22c003a20 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe246a00700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c004870 0x7fe22c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247e00700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe248142840 0x7fe22c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247400700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003a20 0x7fe22c003e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe238009ba0 tx=0x7fe22c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16adafb46f1e8097 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800e6c0 con 0x7fe22c003a20 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23800e820 con 0x7fe22c003a20 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe246000700 1 -- 172.21.15.3:0/436760225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800eb90 con 0x7fe22c003a20 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe24e000700 1 -- 172.21.15.3:0/436760225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003a20 msgr2=0x7fe22c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe24e000700 1 --2- 172.21.15.3:0/436760225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003a20 0x7fe22c003e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe238009ba0 tx=0x7fe22c0058a0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe247e00700 1 -- 172.21.15.3:0/436760225 reap_dead start 2024-09-13T16:54:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe24e000700 1 -- 172.21.15.3:0/436760225 shutdown_connections 2024-09-13T16:54:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe24e000700 1 -- 172.21.15.3:0/436760225 >> 172.21.15.3:0/436760225 conn(0x7fe24806b400 msgr2=0x7fe24806b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe24e000700 1 -- 172.21.15.3:0/436760225 shutdown_connections 2024-09-13T16:54:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.978+0000 7fe24e000700 1 -- 172.21.15.3:0/436760225 wait complete. 2024-09-13T16:54:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 Processor -- start 2024-09-13T16:54:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 -- start start 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe248140650 0x7fe24813f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2480cb2d0 0x7fe2480c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2480c9290 0x7fe24815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe248159b90 con 0x7fe248140650 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24813fcc0 con 0x7fe2480c9290 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24815bad0 con 0x7fe2480cb2d0 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe248140650 0x7fe24813f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe248140650 0x7fe24813f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40366/0 (socket says 172.21.15.3:40366) 2024-09-13T16:54:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 -- 172.21.15.3:0/4245676080 learned_addr learned my addr 172.21.15.3:0/4245676080 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe246a00700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2480cb2d0 0x7fe2480c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247e00700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2480c9290 0x7fe24815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 -- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2480cb2d0 msgr2=0x7fe2480c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2480cb2d0 0x7fe2480c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 -- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2480c9290 msgr2=0x7fe24815bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2480c9290 0x7fe24815bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 -- 172.21.15.3:0/4245676080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24815b1e0 con 0x7fe248140650 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe247400700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe248140650 0x7fe24813f350 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe2380090b0 tx=0x7fe2380047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe23f400700 1 -- 172.21.15.3:0/4245676080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238021070 con 0x7fe248140650 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe23f400700 1 -- 172.21.15.3:0/4245676080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2380045d0 con 0x7fe248140650 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24813f9e0 con 0x7fe248140650 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe23f400700 1 -- 172.21.15.3:0/4245676080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800e6c0 con 0x7fe248140650 2024-09-13T16:54:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.982+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24815b760 con 0x7fe248140650 2024-09-13T16:54:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.986+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248070d60 con 0x7fe248140650 2024-09-13T16:54:00.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.986+0000 7fe23f400700 1 -- 172.21.15.3:0/4245676080 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe238025070 con 0x7fe248140650 2024-09-13T16:54:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.986+0000 7fe23f400700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe228066730 0x7fe228068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.986+0000 7fe23f400700 1 -- 172.21.15.3:0/4245676080 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fe238084f10 con 0x7fe248140650 2024-09-13T16:54:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.990+0000 7fe246a00700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe228066730 0x7fe228068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.990+0000 7fe246a00700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe228066730 0x7fe228068bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe2340011d0 tx=0x7fe234009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:00.994+0000 7fe23f400700 1 -- 172.21.15.3:0/4245676080 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe238052040 con 0x7fe248140650 2024-09-13T16:54:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.222+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2480cc250 con 0x7fe248140650 2024-09-13T16:54:01.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.222+0000 7fe23f400700 1 -- 172.21.15.3:0/4245676080 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fe238017090 con 0x7fe248140650 2024-09-13T16:54:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:01.229 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:54:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe228066730 msgr2=0x7fe228068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe228066730 0x7fe228068bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe2340011d0 tx=0x7fe234009040 comp rx=0 tx=0).stop 2024-09-13T16:54:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe248140650 msgr2=0x7fe24813f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe248140650 0x7fe24813f350 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe2380090b0 tx=0x7fe2380047e0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 shutdown_connections 2024-09-13T16:54:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe228066730 0x7fe228068bf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2480c9290 0x7fe24815bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2480cb2d0 0x7fe2480c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 --2- 172.21.15.3:0/4245676080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe248140650 0x7fe24813f350 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 >> 172.21.15.3:0/4245676080 conn(0x7fe24806b400 msgr2=0x7fe24806b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 shutdown_connections 2024-09-13T16:54:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.226+0000 7fe24e000700 1 -- 172.21.15.3:0/4245676080 wait complete. 2024-09-13T16:54:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:54:01.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:54:01.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f1572600700 1 Processor -- start 2024-09-13T16:54:01.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f1572600700 1 -- start start 2024-09-13T16:54:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f1572600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c06b680 0x7f156c06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f1572600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c06f710 con 0x7f156c06c350 2024-09-13T16:54:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f1572600700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0ce950 con 0x7f156c15d130 2024-09-13T16:54:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f1572600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c06c7e0 con 0x7f156c06b680 2024-09-13T16:54:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f1571c00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f156c06c350 0x7f156c0cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42618/0 (socket says 172.21.15.3:42618) 2024-09-13T16:54:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f1571c00700 1 -- 172.21.15.3:0/463473368 learned_addr learned my addr 172.21.15.3:0/463473368 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:01.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.422+0000 7f156be00700 1 --2- 172.21.15.3:0/463473368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c06b680 0x7f156c06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156b400700 1 -- 172.21.15.3:0/463473368 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122524757 0 0) 0x7f156c06f710 con 0x7f156c06c350 2024-09-13T16:54:01.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156b400700 1 -- 172.21.15.3:0/463473368 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15540035d0 con 0x7f156c06c350 2024-09-13T16:54:01.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156b400700 1 -- 172.21.15.3:0/463473368 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348492971 0 0) 0x7f156c0ce950 con 0x7f156c15d130 2024-09-13T16:54:01.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156b400700 1 -- 172.21.15.3:0/463473368 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c06f710 con 0x7f156c15d130 2024-09-13T16:54:01.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156be00700 1 -- 172.21.15.3:0/463473368 >> v1:172.21.15.157:6789/0 conn(0x7f156c15d130 legacy=0x7f156c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156be00700 1 -- 172.21.15.3:0/463473368 >> v1:172.21.15.3:6789/0 conn(0x7f156c06c350 legacy=0x7f156c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156be00700 1 -- 172.21.15.3:0/463473368 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0cec30 con 0x7f156c06b680 2024-09-13T16:54:01.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156be00700 1 --2- 172.21.15.3:0/463473368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c06b680 0x7f156c06be10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1558009880 tx=0x7f1558009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564c7a27149ea364 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156b400700 1 -- 172.21.15.3:0/463473368 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558011830 con 0x7f156c06b680 2024-09-13T16:54:01.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156b400700 1 -- 172.21.15.3:0/463473368 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1558011990 con 0x7f156c06b680 2024-09-13T16:54:01.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f156b400700 1 -- 172.21.15.3:0/463473368 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558011d20 con 0x7f156c06b680 2024-09-13T16:54:01.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f1572600700 1 -- 172.21.15.3:0/463473368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c06b680 msgr2=0x7f156c06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f1572600700 1 --2- 172.21.15.3:0/463473368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c06b680 0x7f156c06be10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1558009880 tx=0x7f1558009b90 comp rx=0 tx=0).stop 2024-09-13T16:54:01.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f1572600700 1 -- 172.21.15.3:0/463473368 shutdown_connections 2024-09-13T16:54:01.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f1572600700 1 --2- 172.21.15.3:0/463473368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c06b680 0x7f156c06be10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f1572600700 1 -- 172.21.15.3:0/463473368 >> 172.21.15.3:0/463473368 conn(0x7f156c0b24c0 msgr2=0x7f156c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:01.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f1572600700 1 -- 172.21.15.3:0/463473368 shutdown_connections 2024-09-13T16:54:01.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.426+0000 7f1572600700 1 -- 172.21.15.3:0/463473368 wait complete. 2024-09-13T16:54:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1572600700 1 Processor -- start 2024-09-13T16:54:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1572600700 1 -- start start 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1572600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f156c06b680 0x7f156c14bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1572600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f156c06c350 0x7f156c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1572600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c15d130 0x7f156c1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1572600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c14bc60 con 0x7f156c06c350 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1572600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0b64e0 con 0x7f156c06b680 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1572600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0c46a0 con 0x7f156c15d130 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f156be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f156c06c350 0x7f156c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f156be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f156c06c350 0x7f156c151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40376/0 (socket says 172.21.15.3:40376) 2024-09-13T16:54:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f156be00700 1 -- 172.21.15.3:0/2619151570 learned_addr learned my addr 172.21.15.3:0/2619151570 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1571200700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f156c06b680 0x7f156c14bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.430+0000 7f1571c00700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c15d130 0x7f156c1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f156be00700 1 -- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c15d130 msgr2=0x7f156c1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f156be00700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c15d130 0x7f156c1519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f156be00700 1 -- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f156c06b680 msgr2=0x7f156c14bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f156be00700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f156c06b680 0x7f156c14bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f156be00700 1 -- 172.21.15.3:0/2619151570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580094b0 con 0x7f156c06c350 2024-09-13T16:54:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f156be00700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f156c06c350 0x7f156c151470 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1558013040 tx=0x7f1558004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f1568c00700 1 -- 172.21.15.3:0/2619151570 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155801a8c0 con 0x7f156c06c350 2024-09-13T16:54:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f1568c00700 1 -- 172.21.15.3:0/2619151570 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155801e070 con 0x7f156c06c350 2024-09-13T16:54:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f1568c00700 1 -- 172.21.15.3:0/2619151570 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155801ac60 con 0x7f156c06c350 2024-09-13T16:54:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c0c4980 con 0x7f156c06c350 2024-09-13T16:54:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c1521c0 con 0x7f156c06c350 2024-09-13T16:54:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.434+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c0b3270 con 0x7f156c06c350 2024-09-13T16:54:01.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.438+0000 7f1568c00700 1 -- 172.21.15.3:0/2619151570 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f155802d450 con 0x7f156c06c350 2024-09-13T16:54:01.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.438+0000 7f1568c00700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f154c066730 0x7f154c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.438+0000 7f1568c00700 1 -- 172.21.15.3:0/2619151570 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f15580850d0 con 0x7f156c06c350 2024-09-13T16:54:01.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.438+0000 7f1571200700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f154c066730 0x7f154c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.442+0000 7f1571200700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f154c066730 0x7f154c068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f15600011e0 tx=0x7f1560009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.446+0000 7f1568c00700 1 -- 172.21.15.3:0/2619151570 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15580522a0 con 0x7f156c06c350 2024-09-13T16:54:01.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.638+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f156c0c4ea0 con 0x7f156c06c350 2024-09-13T16:54:01.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.642+0000 7f1568c00700 1 -- 172.21.15.3:0/2619151570 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17078 (secure 0 0 0) 0x7f156c0c4ea0 con 0x7f156c06c350 2024-09-13T16:54:01.650 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:01.650 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":64,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:53:15.879157+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:58.182202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:59.098263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6834/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.157:6836/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.3:6835/291442393":"2024-09-14T16:53:08.782245+0000","172.21.15.3:6834/291442393":"2024-09-14T16:53:08.782245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:54:01.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f154c066730 msgr2=0x7f154c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f154c066730 0x7f154c068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f15600011e0 tx=0x7f1560009040 comp rx=0 tx=0).stop 2024-09-13T16:54:01.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f156c06c350 msgr2=0x7f156c151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f156c06c350 0x7f156c151470 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1558013040 tx=0x7f1558004770 comp rx=0 tx=0).stop 2024-09-13T16:54:01.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 shutdown_connections 2024-09-13T16:54:01.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f154c066730 0x7f154c068bf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f156c15d130 0x7f156c1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f156c06c350 0x7f156c151470 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 --2- 172.21.15.3:0/2619151570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f156c06b680 0x7f156c14bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 >> 172.21.15.3:0/2619151570 conn(0x7f156c0b24c0 msgr2=0x7f156c0b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:01.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 shutdown_connections 2024-09-13T16:54:01.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.646+0000 7f1572600700 1 -- 172.21.15.3:0/2619151570 wait complete. 2024-09-13T16:54:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:50:19.278361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:52:58.182202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:52:59.098263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:54:01.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:54:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.854+0000 7f6789a00700 1 Processor -- start 2024-09-13T16:54:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.854+0000 7f6789a00700 1 -- start start 2024-09-13T16:54:01.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.854+0000 7f6789a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406c520 0x7f6784167e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.854+0000 7f6789a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67840d0720 con 0x7f678406b680 2024-09-13T16:54:01.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.854+0000 7f6789a00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678406f710 con 0x7f678415d130 2024-09-13T16:54:01.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.854+0000 7f6789a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67840ce950 con 0x7f678406c520 2024-09-13T16:54:01.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.854+0000 7f6783400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f678406b680 0x7f678406be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42628/0 (socket says 172.21.15.3:42628) 2024-09-13T16:54:01.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.854+0000 7f6783400700 1 -- 172.21.15.3:0/1151898031 learned_addr learned my addr 172.21.15.3:0/1151898031 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:01.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789000700 1 --2- 172.21.15.3:0/1151898031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406c520 0x7f6784167e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6782a00700 1 -- 172.21.15.3:0/1151898031 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221148693 0 0) 0x7f678406f710 con 0x7f678415d130 2024-09-13T16:54:01.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6782a00700 1 -- 172.21.15.3:0/1151898031 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67680035d0 con 0x7f678415d130 2024-09-13T16:54:01.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789000700 1 -- 172.21.15.3:0/1151898031 >> v1:172.21.15.157:6789/0 conn(0x7f678415d130 legacy=0x7f678406eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789000700 1 -- 172.21.15.3:0/1151898031 >> v1:172.21.15.3:6789/0 conn(0x7f678406b680 legacy=0x7f678406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789000700 1 -- 172.21.15.3:0/1151898031 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840cec30 con 0x7f678406c520 2024-09-13T16:54:01.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789000700 1 --2- 172.21.15.3:0/1151898031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406c520 0x7f6784167e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f677c009dc0 tx=0x7f677c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e70b3fdb8c0b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6782a00700 1 -- 172.21.15.3:0/1151898031 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c00e070 con 0x7f678406c520 2024-09-13T16:54:01.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6782a00700 1 -- 172.21.15.3:0/1151898031 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c00bb20 con 0x7f678406c520 2024-09-13T16:54:01.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6782a00700 1 -- 172.21.15.3:0/1151898031 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c012430 con 0x7f678406c520 2024-09-13T16:54:01.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789a00700 1 -- 172.21.15.3:0/1151898031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406c520 msgr2=0x7f6784167e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789a00700 1 --2- 172.21.15.3:0/1151898031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406c520 0x7f6784167e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f677c009dc0 tx=0x7f677c00b330 comp rx=0 tx=0).stop 2024-09-13T16:54:01.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789a00700 1 -- 172.21.15.3:0/1151898031 shutdown_connections 2024-09-13T16:54:01.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789a00700 1 --2- 172.21.15.3:0/1151898031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406c520 0x7f6784167e30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.858+0000 7f6789a00700 1 -- 172.21.15.3:0/1151898031 >> 172.21.15.3:0/1151898031 conn(0x7f67840b24c0 msgr2=0x7f67840b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:01.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.862+0000 7f6789a00700 1 -- 172.21.15.3:0/1151898031 shutdown_connections 2024-09-13T16:54:01.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.862+0000 7f6789a00700 1 -- 172.21.15.3:0/1151898031 wait complete. 2024-09-13T16:54:01.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.862+0000 7f6789a00700 1 Processor -- start 2024-09-13T16:54:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6789a00700 1 -- start start 2024-09-13T16:54:01.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6789a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678406b680 0x7f67840cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6789a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678406c520 0x7f67840cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6789a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678415d130 0x7f67840b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6789a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67840cb750 con 0x7f678406c520 2024-09-13T16:54:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6789a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67840cb520 con 0x7f678406b680 2024-09-13T16:54:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6789a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67840cd7b0 con 0x7f678415d130 2024-09-13T16:54:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678406c520 0x7f67840cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678406c520 0x7f67840cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40388/0 (socket says 172.21.15.3:40388) 2024-09-13T16:54:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 -- 172.21.15.3:0/2874124969 learned_addr learned my addr 172.21.15.3:0/2874124969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783e00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678406b680 0x7f67840cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6789000700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678415d130 0x7f67840b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 -- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678415d130 msgr2=0x7f67840b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678415d130 0x7f67840b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 -- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678406b680 msgr2=0x7f67840cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678406b680 0x7f67840cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 -- 172.21.15.3:0/2874124969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c009a70 con 0x7f678406c520 2024-09-13T16:54:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.866+0000 7f6783400700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678406c520 0x7f67840cafe0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f677000c500 tx=0x7f677000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.870+0000 7f677be00700 1 -- 172.21.15.3:0/2874124969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770005d90 con 0x7f678406c520 2024-09-13T16:54:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.870+0000 7f677be00700 1 -- 172.21.15.3:0/2874124969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67700070d0 con 0x7f678406c520 2024-09-13T16:54:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.870+0000 7f677be00700 1 -- 172.21.15.3:0/2874124969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770007440 con 0x7f678406c520 2024-09-13T16:54:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.870+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67840ccbe0 con 0x7f678406c520 2024-09-13T16:54:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.870+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67840cd160 con 0x7f678406c520 2024-09-13T16:54:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.870+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67840cd8f0 con 0x7f678406c520 2024-09-13T16:54:01.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.870+0000 7f677be00700 1 -- 172.21.15.3:0/2874124969 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f67700075a0 con 0x7f678406c520 2024-09-13T16:54:01.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.874+0000 7f677be00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6764066760 0x7f6764068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.874+0000 7f677be00700 1 -- 172.21.15.3:0/2874124969 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f67700835a0 con 0x7f678406c520 2024-09-13T16:54:01.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.874+0000 7f6783e00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6764066760 0x7f6764068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.874+0000 7f6783e00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6764066760 0x7f6764068c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f67740011e0 tx=0x7f6774009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:01.882+0000 7f677be00700 1 -- 172.21.15.3:0/2874124969 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6770050780 con 0x7f678406c520 2024-09-13T16:54:02.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.062+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67840b6930 con 0x7f678406c520 2024-09-13T16:54:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.062+0000 7f677be00700 1 -- 172.21.15.3:0/2874124969 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6770053da0 con 0x7f678406c520 2024-09-13T16:54:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-13T16:54:02.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6764066760 msgr2=0x7f6764068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6764066760 0x7f6764068c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f67740011e0 tx=0x7f6774009040 comp rx=0 tx=0).stop 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678406c520 msgr2=0x7f67840cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678406c520 0x7f67840cafe0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f677000c500 tx=0x7f677000c810 comp rx=0 tx=0).stop 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 shutdown_connections 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f6764066760 0x7f6764068c20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678415d130 0x7f67840b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f678406c520 0x7f67840cafe0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 --2- 172.21.15.3:0/2874124969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678406b680 0x7f67840cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 >> 172.21.15.3:0/2874124969 conn(0x7f67840b24c0 msgr2=0x7f67840c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 shutdown_connections 2024-09-13T16:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.066+0000 7f6789a00700 1 -- 172.21.15.3:0/2874124969 wait complete. 2024-09-13T16:54:02.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:54:02.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.266+0000 7fcd1ca00700 1 Processor -- start 2024-09-13T16:54:02.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.266+0000 7fcd1ca00700 1 -- start start 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd1ca00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd18160f30 0x7fcd18165770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd1ca00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806d640 0x7fcd18164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd1ca00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1806ba80 con 0x7fcd1806da80 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd1ca00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1806fad0 con 0x7fcd18160f30 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd1ca00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1806bcd0 con 0x7fcd1806d640 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd16800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd18160f30 0x7fcd18165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd15e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806d640 0x7fcd18164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd16800700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd18160f30 0x7fcd18165770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:45582/0 (socket says 172.21.15.3:45582) 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd16800700 1 -- 172.21.15.3:0/3251166494 learned_addr learned my addr 172.21.15.3:0/3251166494 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:02.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd17200700 1 --1- 172.21.15.3:0/3251166494 >> v1:172.21.15.157:6789/0 conn(0x7fcd1806da80 0x7fcd180b1690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:43852/0 (socket says 172.21.15.3:43852) 2024-09-13T16:54:02.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd16800700 1 -- 172.21.15.3:0/3251166494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806d640 msgr2=0x7fcd18164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:02.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd16800700 1 --2- 172.21.15.3:0/3251166494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806d640 0x7fcd18164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd16800700 1 -- 172.21.15.3:0/3251166494 >> v1:172.21.15.157:6789/0 conn(0x7fcd1806da80 legacy=0x7fcd180b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd16800700 1 -- 172.21.15.3:0/3251166494 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1806bfb0 con 0x7fcd18160f30 2024-09-13T16:54:02.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.270+0000 7fcd16800700 1 --2- 172.21.15.3:0/3251166494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 0x7fcd18165770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009910 tx=0x7fcd08009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e948600b7814e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:02.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd15400700 1 -- 172.21.15.3:0/3251166494 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08011830 con 0x7fcd18160f30 2024-09-13T16:54:02.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd15400700 1 -- 172.21.15.3:0/3251166494 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08011990 con 0x7fcd18160f30 2024-09-13T16:54:02.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd15400700 1 -- 172.21.15.3:0/3251166494 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08011d40 con 0x7fcd18160f30 2024-09-13T16:54:02.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd1ca00700 1 -- 172.21.15.3:0/3251166494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 msgr2=0x7fcd18165770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/3251166494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 0x7fcd18165770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009910 tx=0x7fcd08009c20 comp rx=0 tx=0).stop 2024-09-13T16:54:02.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd1ca00700 1 -- 172.21.15.3:0/3251166494 shutdown_connections 2024-09-13T16:54:02.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/3251166494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806d640 0x7fcd18164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/3251166494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 0x7fcd18165770 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd1ca00700 1 -- 172.21.15.3:0/3251166494 >> 172.21.15.3:0/3251166494 conn(0x7fcd1806a2f0 msgr2=0x7fcd1806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:02.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd1ca00700 1 -- 172.21.15.3:0/3251166494 shutdown_connections 2024-09-13T16:54:02.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.274+0000 7fcd1ca00700 1 -- 172.21.15.3:0/3251166494 wait complete. 2024-09-13T16:54:02.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd1ca00700 1 Processor -- start 2024-09-13T16:54:02.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd1ca00700 1 -- start start 2024-09-13T16:54:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd1ca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1806d640 0x7fcd1814c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd1ca00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806da80 0x7fcd180af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd1ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 0x7fcd180af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd1ca00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18152d60 con 0x7fcd1806d640 2024-09-13T16:54:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd1ca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18153040 con 0x7fcd18160f30 2024-09-13T16:54:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd1ca00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18153320 con 0x7fcd1806da80 2024-09-13T16:54:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1806d640 0x7fcd1814c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1806d640 0x7fcd1814c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40398/0 (socket says 172.21.15.3:40398) 2024-09-13T16:54:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 -- 172.21.15.3:0/1974244657 learned_addr learned my addr 172.21.15.3:0/1974244657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd15e00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806da80 0x7fcd180af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd17200700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 0x7fcd180af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 -- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806da80 msgr2=0x7fcd180af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806da80 0x7fcd180af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 -- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 msgr2=0x7fcd180af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 0x7fcd180af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 -- 172.21.15.3:0/1974244657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd08009570 con 0x7fcd1806d640 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd17200700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 0x7fcd180af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.278+0000 7fcd16800700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1806d640 0x7fcd1814c030 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcd18072390 tx=0x7fcd08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.282+0000 7fcd0ea00700 1 -- 172.21.15.3:0/1974244657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08004510 con 0x7fcd1806d640 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.282+0000 7fcd0ea00700 1 -- 172.21.15.3:0/1974244657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0801e070 con 0x7fcd1806d640 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.282+0000 7fcd0ea00700 1 -- 172.21.15.3:0/1974244657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08003900 con 0x7fcd1806d640 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.282+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1814d700 con 0x7fcd1806d640 2024-09-13T16:54:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.282+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1814cd10 con 0x7fcd1806d640 2024-09-13T16:54:02.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.282+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1814dc20 con 0x7fcd1806d640 2024-09-13T16:54:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.286+0000 7fcd0ea00700 1 -- 172.21.15.3:0/1974244657 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcd08003a60 con 0x7fcd1806d640 2024-09-13T16:54:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.286+0000 7fcd0ea00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccf8066730 0x7fccf8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.286+0000 7fcd0ea00700 1 -- 172.21.15.3:0/1974244657 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fcd08084fb0 con 0x7fcd1806d640 2024-09-13T16:54:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.286+0000 7fcd15e00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccf8066730 0x7fccf8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.290+0000 7fcd15e00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccf8066730 0x7fccf8068bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcd18154000 tx=0x7fcd04009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:02.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.294+0000 7fcd0ea00700 1 -- 172.21.15.3:0/1974244657 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcd080b6db0 con 0x7fcd1806d640 2024-09-13T16:54:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.474+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd18164940 con 0x7fcd1806d640 2024-09-13T16:54:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.474+0000 7fcd0ea00700 1 -- 172.21.15.3:0/1974244657 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd08020020 con 0x7fcd1806d640 2024-09-13T16:54:02.482 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-13T16:54:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccf8066730 msgr2=0x7fccf8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccf8066730 0x7fccf8068bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcd18154000 tx=0x7fcd04009240 comp rx=0 tx=0).stop 2024-09-13T16:54:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1806d640 msgr2=0x7fcd1814c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1806d640 0x7fcd1814c030 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcd18072390 tx=0x7fcd08004770 comp rx=0 tx=0).stop 2024-09-13T16:54:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 shutdown_connections 2024-09-13T16:54:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fccf8066730 0x7fccf8068bf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd18160f30 0x7fcd180af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd1806da80 0x7fcd180af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 --2- 172.21.15.3:0/1974244657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1806d640 0x7fcd1814c030 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 >> 172.21.15.3:0/1974244657 conn(0x7fcd1806a2f0 msgr2=0x7fcd180b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 shutdown_connections 2024-09-13T16:54:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.478+0000 7fcd1ca00700 1 -- 172.21.15.3:0/1974244657 wait complete. 2024-09-13T16:54:02.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:54:02.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.686+0000 7ff0a6000700 1 Processor -- start 2024-09-13T16:54:02.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.686+0000 7ff0a6000700 1 -- start start 2024-09-13T16:54:02.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff0a6000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0a0161ff0 0x7ff0a006fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff0a6000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a00cc8f0 con 0x7ff0a0144750 2024-09-13T16:54:02.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff0a6000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a0165b10 con 0x7ff0a01623d0 2024-09-13T16:54:02.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff0a6000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a01472c0 con 0x7ff0a0161ff0 2024-09-13T16:54:02.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff09ea00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff0a0144750 0x7ff0a0164cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42632/0 (socket says 172.21.15.3:42632) 2024-09-13T16:54:02.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff09ea00700 1 -- 172.21.15.3:0/3481961024 learned_addr learned my addr 172.21.15.3:0/3481961024 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:02.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff09f400700 1 --2- 172.21.15.3:0/3481961024 >> v2:172.21.15.157:3300/0 conn(0x7ff0a0161ff0 0x7ff0a006fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff09e000700 1 -- 172.21.15.3:0/3481961024 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089498934 0 0) 0x7ff0a00cc8f0 con 0x7ff0a0144750 2024-09-13T16:54:02.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff09e000700 1 -- 172.21.15.3:0/3481961024 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0840035d0 con 0x7ff0a0144750 2024-09-13T16:54:02.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff09f400700 1 -- 172.21.15.3:0/3481961024 >> v1:172.21.15.157:6789/0 conn(0x7ff0a01623d0 legacy=0x7ff0a0165400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff09f400700 1 -- 172.21.15.3:0/3481961024 >> v1:172.21.15.3:6789/0 conn(0x7ff0a0144750 legacy=0x7ff0a0164cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.690+0000 7ff09f400700 1 -- 172.21.15.3:0/3481961024 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a01475a0 con 0x7ff0a0161ff0 2024-09-13T16:54:02.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff09f400700 1 --2- 172.21.15.3:0/3481961024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0161ff0 0x7ff0a006fde0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff090009850 tx=0x7ff090009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f986c89019bd970 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:02.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff09e000700 1 -- 172.21.15.3:0/3481961024 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090011830 con 0x7ff0a0161ff0 2024-09-13T16:54:02.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff09e000700 1 -- 172.21.15.3:0/3481961024 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090011990 con 0x7ff0a0161ff0 2024-09-13T16:54:02.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff09e000700 1 -- 172.21.15.3:0/3481961024 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090011d40 con 0x7ff0a0161ff0 2024-09-13T16:54:02.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff0a6000700 1 -- 172.21.15.3:0/3481961024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0161ff0 msgr2=0x7ff0a006fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff0a6000700 1 --2- 172.21.15.3:0/3481961024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0161ff0 0x7ff0a006fde0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff090009850 tx=0x7ff090009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:02.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff0a6000700 1 -- 172.21.15.3:0/3481961024 shutdown_connections 2024-09-13T16:54:02.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff0a6000700 1 --2- 172.21.15.3:0/3481961024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0161ff0 0x7ff0a006fde0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff0a6000700 1 -- 172.21.15.3:0/3481961024 >> 172.21.15.3:0/3481961024 conn(0x7ff0a00b07a0 msgr2=0x7ff0a00b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:02.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff0a6000700 1 -- 172.21.15.3:0/3481961024 shutdown_connections 2024-09-13T16:54:02.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.694+0000 7ff0a6000700 1 -- 172.21.15.3:0/3481961024 wait complete. 2024-09-13T16:54:02.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 Processor -- start 2024-09-13T16:54:02.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 -- start start 2024-09-13T16:54:02.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0144750 0x7ff0a0146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0161ff0 0x7ff0a01430f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a01623d0 0x7ff0a015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0146fa0 con 0x7ff0a0161ff0 2024-09-13T16:54:02.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0158790 con 0x7ff0a0144750 2024-09-13T16:54:02.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a015d4d0 con 0x7ff0a01623d0 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0161ff0 0x7ff0a01430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0161ff0 0x7ff0a01430f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40406/0 (socket says 172.21.15.3:40406) 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 -- 172.21.15.3:0/441641779 learned_addr learned my addr 172.21.15.3:0/441641779 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09f400700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0144750 0x7ff0a0146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09fe00700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a01623d0 0x7ff0a015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 -- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a01623d0 msgr2=0x7ff0a015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a01623d0 0x7ff0a015a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 -- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0144750 msgr2=0x7ff0a0146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0144750 0x7ff0a0146910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 -- 172.21.15.3:0/441641779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900094b0 con 0x7ff0a0161ff0 2024-09-13T16:54:02.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09fe00700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a01623d0 0x7ff0a015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:54:02.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff09ea00700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0161ff0 0x7ff0a01430f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c700 tx=0x7ff08c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:02.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff097400700 1 -- 172.21.15.3:0/441641779 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c005d90 con 0x7ff0a0161ff0 2024-09-13T16:54:02.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff097400700 1 -- 172.21.15.3:0/441641779 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c0070d0 con 0x7ff0a0161ff0 2024-09-13T16:54:02.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a015d7b0 con 0x7ff0a0161ff0 2024-09-13T16:54:02.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.698+0000 7ff097400700 1 -- 172.21.15.3:0/441641779 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c007480 con 0x7ff0a0161ff0 2024-09-13T16:54:02.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.702+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a015e230 con 0x7ff0a0161ff0 2024-09-13T16:54:02.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.702+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a0070650 con 0x7ff0a0161ff0 2024-09-13T16:54:02.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.702+0000 7ff097400700 1 -- 172.21.15.3:0/441641779 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff08c0075e0 con 0x7ff0a0161ff0 2024-09-13T16:54:02.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.706+0000 7ff097400700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff080066730 0x7ff080068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:02.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.706+0000 7ff097400700 1 -- 172.21.15.3:0/441641779 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff08c083a70 con 0x7ff0a0161ff0 2024-09-13T16:54:02.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.706+0000 7ff09f400700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff080066730 0x7ff080068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:02.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.706+0000 7ff09f400700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff080066730 0x7ff080068bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff0900090b0 tx=0x7ff090008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.710+0000 7ff097400700 1 -- 172.21.15.3:0/441641779 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff08c050b20 con 0x7ff0a0161ff0 2024-09-13T16:54:02.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.898+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0a0163e50 con 0x7ff0a0161ff0 2024-09-13T16:54:02.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.898+0000 7ff097400700 1 -- 172.21.15.3:0/441641779 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff08c054140 con 0x7ff0a0161ff0 2024-09-13T16:54:02.905 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-13T16:54:02.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff080066730 msgr2=0x7ff080068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff080066730 0x7ff080068bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff0900090b0 tx=0x7ff090008030 comp rx=0 tx=0).stop 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0161ff0 msgr2=0x7ff0a01430f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0161ff0 0x7ff0a01430f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c700 tx=0x7ff08c00cac0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 shutdown_connections 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff080066730 0x7ff080068bf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a01623d0 0x7ff0a015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0161ff0 0x7ff0a01430f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 --2- 172.21.15.3:0/441641779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0144750 0x7ff0a0146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 >> 172.21.15.3:0/441641779 conn(0x7ff0a00b07a0 msgr2=0x7ff0a01607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 shutdown_connections 2024-09-13T16:54:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:02.902+0000 7ff0a6000700 1 -- 172.21.15.3:0/441641779 wait complete. 2024-09-13T16:54:02.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:03.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.094+0000 7fefdf600700 1 Processor -- start 2024-09-13T16:54:03.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.094+0000 7fefdf600700 1 -- start start 2024-09-13T16:54:03.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefdf600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefd81444c0 0x7fefd815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefdf600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefd8071600 0x7fefd806bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefdf600700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd8071170 con 0x7fefd806fff0 2024-09-13T16:54:03.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefdf600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd81448a0 con 0x7fefd8071600 2024-09-13T16:54:03.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefdf600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd815c140 con 0x7fefd81444c0 2024-09-13T16:54:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefd8071600 0x7fefd806bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefd8071600 0x7fefd806bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40414/0 (socket says 172.21.15.3:40414) 2024-09-13T16:54:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7e00700 1 -- 172.21.15.3:0/3008465369 learned_addr learned my addr 172.21.15.3:0/3008465369 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefdd600700 1 --1- 172.21.15.3:0/3008465369 >> v1:172.21.15.3:6789/0 conn(0x7fefd806fff0 0x7fefd815e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42648/0 (socket says 172.21.15.3:42648) 2024-09-13T16:54:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefdcc00700 1 --2- 172.21.15.3:0/3008465369 >> v2:172.21.15.157:3300/0 conn(0x7fefd81444c0 0x7fefd815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7e00700 1 -- 172.21.15.3:0/3008465369 >> v2:172.21.15.157:3300/0 conn(0x7fefd81444c0 msgr2=0x7fefd815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7e00700 1 --2- 172.21.15.3:0/3008465369 >> v2:172.21.15.157:3300/0 conn(0x7fefd81444c0 0x7fefd815bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7e00700 1 -- 172.21.15.3:0/3008465369 >> v1:172.21.15.3:6789/0 conn(0x7fefd806fff0 legacy=0x7fefd815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7e00700 1 -- 172.21.15.3:0/3008465369 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8070560 con 0x7fefd8071600 2024-09-13T16:54:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7e00700 1 --2- 172.21.15.3:0/3008465369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd8071600 0x7fefd806bb60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fefd8072b30 tx=0x7fefc4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2668841d95d1f3b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7400700 1 -- 172.21.15.3:0/3008465369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400b8a0 con 0x7fefd8071600 2024-09-13T16:54:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7400700 1 -- 172.21.15.3:0/3008465369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc400ba00 con 0x7fefd8071600 2024-09-13T16:54:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.098+0000 7fefd7400700 1 -- 172.21.15.3:0/3008465369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400bd90 con 0x7fefd8071600 2024-09-13T16:54:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.102+0000 7fefdf600700 1 -- 172.21.15.3:0/3008465369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd8071600 msgr2=0x7fefd806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.102+0000 7fefdf600700 1 --2- 172.21.15.3:0/3008465369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd8071600 0x7fefd806bb60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fefd8072b30 tx=0x7fefc4009bf0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.102+0000 7fefdf600700 1 -- 172.21.15.3:0/3008465369 shutdown_connections 2024-09-13T16:54:03.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.102+0000 7fefdf600700 1 --2- 172.21.15.3:0/3008465369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd8071600 0x7fefd806bb60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.102+0000 7fefdf600700 1 --2- 172.21.15.3:0/3008465369 >> v2:172.21.15.157:3300/0 conn(0x7fefd81444c0 0x7fefd815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.102+0000 7fefdf600700 1 -- 172.21.15.3:0/3008465369 >> 172.21.15.3:0/3008465369 conn(0x7fefd80b07a0 msgr2=0x7fefd80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:03.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.102+0000 7fefdf600700 1 -- 172.21.15.3:0/3008465369 shutdown_connections 2024-09-13T16:54:03.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.102+0000 7fefdf600700 1 -- 172.21.15.3:0/3008465369 wait complete. 2024-09-13T16:54:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdf600700 1 Processor -- start 2024-09-13T16:54:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdf600700 1 -- start start 2024-09-13T16:54:03.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdf600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd806fff0 0x7fefd80b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdf600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefd8071600 0x7fefd813bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdf600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd81444c0 0x7fefd815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdf600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd813b870 con 0x7fefd81444c0 2024-09-13T16:54:03.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdf600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd813f340 con 0x7fefd806fff0 2024-09-13T16:54:03.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdf600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd813f620 con 0x7fefd8071600 2024-09-13T16:54:03.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdd600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd81444c0 0x7fefd815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdd600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd81444c0 0x7fefd815a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40426/0 (socket says 172.21.15.3:40426) 2024-09-13T16:54:03.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdd600700 1 -- 172.21.15.3:0/2505270043 learned_addr learned my addr 172.21.15.3:0/2505270043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:03.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdcc00700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd806fff0 0x7fefd80b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefd7e00700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefd8071600 0x7fefd813bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdd600700 1 -- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefd8071600 msgr2=0x7fefd813bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdd600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefd8071600 0x7fefd813bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdd600700 1 -- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd806fff0 msgr2=0x7fefd80b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdd600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd806fff0 0x7fefd80b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.106+0000 7fefdd600700 1 -- 172.21.15.3:0/2505270043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4009570 con 0x7fefd81444c0 2024-09-13T16:54:03.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.110+0000 7fefdd600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd81444c0 0x7fefd815a1b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fefd000cb50 tx=0x7fefd000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.110+0000 7fefd4c00700 1 -- 172.21.15.3:0/2505270043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000a9e0 con 0x7fefd81444c0 2024-09-13T16:54:03.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.110+0000 7fefd4c00700 1 -- 172.21.15.3:0/2505270043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd000ab40 con 0x7fefd81444c0 2024-09-13T16:54:03.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.110+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd813f900 con 0x7fefd81444c0 2024-09-13T16:54:03.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.110+0000 7fefd4c00700 1 -- 172.21.15.3:0/2505270043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0007070 con 0x7fefd81444c0 2024-09-13T16:54:03.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.110+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd813fec0 con 0x7fefd81444c0 2024-09-13T16:54:03.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.110+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd81400b0 con 0x7fefd81444c0 2024-09-13T16:54:03.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.114+0000 7fefd4c00700 1 -- 172.21.15.3:0/2505270043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fefd00071d0 con 0x7fefd81444c0 2024-09-13T16:54:03.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.114+0000 7fefd4c00700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fefb8066730 0x7fefb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.114+0000 7fefd4c00700 1 -- 172.21.15.3:0/2505270043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fefd0084170 con 0x7fefd81444c0 2024-09-13T16:54:03.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.114+0000 7fefdcc00700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fefb8066730 0x7fefb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.114+0000 7fefdcc00700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fefb8066730 0x7fefb8068bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fefcc007a60 tx=0x7fefcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.118+0000 7fefd4c00700 1 -- 172.21.15.3:0/2505270043 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fefd00512b0 con 0x7fefd81444c0 2024-09-13T16:54:03.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.346+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefd806c460 con 0x7fefd81444c0 2024-09-13T16:54:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.346+0000 7fefd4c00700 1 -- 172.21.15.3:0/2505270043 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fefd00548d0 con 0x7fefd81444c0 2024-09-13T16:54:03.354 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:54:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fefb8066730 msgr2=0x7fefb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fefb8066730 0x7fefb8068bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fefcc007a60 tx=0x7fefcc008040 comp rx=0 tx=0).stop 2024-09-13T16:54:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd81444c0 msgr2=0x7fefd815a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd81444c0 0x7fefd815a1b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fefd000cb50 tx=0x7fefd000ce60 comp rx=0 tx=0).stop 2024-09-13T16:54:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 shutdown_connections 2024-09-13T16:54:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fefb8066730 0x7fefb8068bf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd81444c0 0x7fefd815a1b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefd8071600 0x7fefd813bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 --2- 172.21.15.3:0/2505270043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd806fff0 0x7fefd80b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 >> 172.21.15.3:0/2505270043 conn(0x7fefd80b07a0 msgr2=0x7fefd80ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 shutdown_connections 2024-09-13T16:54:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.350+0000 7fefdf600700 1 -- 172.21.15.3:0/2505270043 wait complete. 2024-09-13T16:54:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:54:03.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:54:03.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.546+0000 7f4fef000700 1 Processor -- start 2024-09-13T16:54:03.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.546+0000 7f4fef000700 1 -- start start 2024-09-13T16:54:03.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fef000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe8165770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fef000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe806ba80 con 0x7f4fe806d640 2024-09-13T16:54:03.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fef000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe806fad0 con 0x7f4fe806da80 2024-09-13T16:54:03.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fef000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe806bcd0 con 0x7f4fe8160f30 2024-09-13T16:54:03.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fed200700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4fe806d640 0x7f4fe8164bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42652/0 (socket says 172.21.15.3:42652) 2024-09-13T16:54:03.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fed200700 1 -- 172.21.15.3:0/486054293 learned_addr learned my addr 172.21.15.3:0/486054293 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:03.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fedc00700 1 --2- 172.21.15.3:0/486054293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe8165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fe7e00700 1 -- 172.21.15.3:0/486054293 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800175249 0 0) 0x7f4fe806ba80 con 0x7f4fe806d640 2024-09-13T16:54:03.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fe7e00700 1 -- 172.21.15.3:0/486054293 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd00035d0 con 0x7f4fe806d640 2024-09-13T16:54:03.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fe7e00700 1 -- 172.21.15.3:0/486054293 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014694934 0 0) 0x7f4fe806fad0 con 0x7f4fe806da80 2024-09-13T16:54:03.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fe7e00700 1 -- 172.21.15.3:0/486054293 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe806ba80 con 0x7f4fe806da80 2024-09-13T16:54:03.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fedc00700 1 -- 172.21.15.3:0/486054293 >> v1:172.21.15.157:6789/0 conn(0x7f4fe806da80 legacy=0x7f4fe8070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fedc00700 1 -- 172.21.15.3:0/486054293 >> v1:172.21.15.3:6789/0 conn(0x7f4fe806d640 legacy=0x7f4fe8164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fedc00700 1 -- 172.21.15.3:0/486054293 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe806bfb0 con 0x7f4fe8160f30 2024-09-13T16:54:03.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.550+0000 7f4fedc00700 1 --2- 172.21.15.3:0/486054293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe8165770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009880 tx=0x7f4fdc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d83451a7478d6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fe7e00700 1 -- 172.21.15.3:0/486054293 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc011830 con 0x7f4fe8160f30 2024-09-13T16:54:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fe7e00700 1 -- 172.21.15.3:0/486054293 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc011990 con 0x7f4fe8160f30 2024-09-13T16:54:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fe7e00700 1 -- 172.21.15.3:0/486054293 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc011d20 con 0x7f4fe8160f30 2024-09-13T16:54:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fef000700 1 -- 172.21.15.3:0/486054293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 msgr2=0x7f4fe8165770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fef000700 1 --2- 172.21.15.3:0/486054293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe8165770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009880 tx=0x7f4fdc009b90 comp rx=0 tx=0).stop 2024-09-13T16:54:03.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fef000700 1 -- 172.21.15.3:0/486054293 shutdown_connections 2024-09-13T16:54:03.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fef000700 1 --2- 172.21.15.3:0/486054293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe8165770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fef000700 1 -- 172.21.15.3:0/486054293 >> 172.21.15.3:0/486054293 conn(0x7f4fe806a2f0 msgr2=0x7f4fe806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:03.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fef000700 1 -- 172.21.15.3:0/486054293 shutdown_connections 2024-09-13T16:54:03.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fef000700 1 -- 172.21.15.3:0/486054293 wait complete. 2024-09-13T16:54:03.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.554+0000 7f4fef000700 1 Processor -- start 2024-09-13T16:54:03.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fef000700 1 -- start start 2024-09-13T16:54:03.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fef000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fe806d640 0x7f4fe814cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fef000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fe806da80 0x7f4fe814e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fef000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fef000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe814e830 con 0x7f4fe806da80 2024-09-13T16:54:03.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fef000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe81f7490 con 0x7f4fe806d640 2024-09-13T16:54:03.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fef000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe814ead0 con 0x7f4fe8160f30 2024-09-13T16:54:03.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fe806da80 0x7f4fe814e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fe806da80 0x7f4fe814e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40442/0 (socket says 172.21.15.3:40442) 2024-09-13T16:54:03.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 -- 172.21.15.3:0/2432910323 learned_addr learned my addr 172.21.15.3:0/2432910323 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:03.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fedc00700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fe806d640 0x7f4fe814cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fee600700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 -- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 msgr2=0x7f4fe815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 -- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fe806d640 msgr2=0x7f4fe814cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fe806d640 0x7f4fe814cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 -- 172.21.15.3:0/2432910323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0094b0 con 0x7f4fe806da80 2024-09-13T16:54:03.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fed200700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fe806da80 0x7f4fe814e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c560 tx=0x7f4fd400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fe5600700 1 -- 172.21.15.3:0/2432910323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4005d90 con 0x7f4fe806da80 2024-09-13T16:54:03.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.558+0000 7f4fe5600700 1 -- 172.21.15.3:0/2432910323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd40070d0 con 0x7f4fe806da80 2024-09-13T16:54:03.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.562+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe814edb0 con 0x7f4fe806da80 2024-09-13T16:54:03.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.562+0000 7f4fe5600700 1 -- 172.21.15.3:0/2432910323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4007460 con 0x7f4fe806da80 2024-09-13T16:54:03.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.562+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe81f7750 con 0x7f4fe806da80 2024-09-13T16:54:03.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.562+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe8165770 con 0x7f4fe806da80 2024-09-13T16:54:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.562+0000 7f4fe5600700 1 -- 172.21.15.3:0/2432910323 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4fd40075c0 con 0x7f4fe806da80 2024-09-13T16:54:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.566+0000 7f4fe5600700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4fc8066730 0x7f4fc8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.566+0000 7f4fe5600700 1 -- 172.21.15.3:0/2432910323 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f4fd40838e0 con 0x7f4fe806da80 2024-09-13T16:54:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.566+0000 7f4fedc00700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4fc8066730 0x7f4fc8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.566+0000 7f4fedc00700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4fc8066730 0x7f4fc8068bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0090b0 tx=0x7f4fdc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.570+0000 7f4fe5600700 1 -- 172.21.15.3:0/2432910323 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4fd4050a10 con 0x7f4fe806da80 2024-09-13T16:54:03.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.766+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fe80715c0 con 0x7f4fe806da80 2024-09-13T16:54:03.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.766+0000 7f4fe5600700 1 -- 172.21.15.3:0/2432910323 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17078 (secure 0 0 0) 0x7f4fd4054030 con 0x7f4fe806da80 2024-09-13T16:54:03.776 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:03.776 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":64,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:53:15.879157+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:58.182202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:59.098263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6834/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.157:6836/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.3:6835/291442393":"2024-09-14T16:53:08.782245+0000","172.21.15.3:6834/291442393":"2024-09-14T16:53:08.782245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:54:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4fc8066730 msgr2=0x7f4fc8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4fc8066730 0x7f4fc8068bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0090b0 tx=0x7f4fdc01c040 comp rx=0 tx=0).stop 2024-09-13T16:54:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fe806da80 msgr2=0x7f4fe814e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fe806da80 0x7f4fe814e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c560 tx=0x7f4fd400c870 comp rx=0 tx=0).stop 2024-09-13T16:54:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 shutdown_connections 2024-09-13T16:54:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f4fc8066730 0x7f4fc8068bf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fe8160f30 0x7f4fe815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fe806da80 0x7f4fe814e2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 --2- 172.21.15.3:0/2432910323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fe806d640 0x7f4fe814cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 >> 172.21.15.3:0/2432910323 conn(0x7f4fe806a2f0 msgr2=0x7f4fe8161ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:03.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 shutdown_connections 2024-09-13T16:54:03.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.770+0000 7f4fef000700 1 -- 172.21.15.3:0/2432910323 wait complete. 2024-09-13T16:54:03.802 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T16:54:03.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3e00700 1 Processor -- start 2024-09-13T16:54:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3e00700 1 -- start start 2024-09-13T16:54:03.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415cc40 0x7f15a406fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15a406d540 0x7f15a40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15a415d020 0x7f15a40ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a40adc60 con 0x7f15a415d020 2024-09-13T16:54:03.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a406da50 con 0x7f15a406d540 2024-09-13T16:54:03.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a406dd30 con 0x7f15a415cc40 2024-09-13T16:54:03.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15a415d020 0x7f15a40ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15a415d020 0x7f15a40ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38692/0 (socket says 172.21.15.3:38692) 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 -- 172.21.15.3:0/1428640592 learned_addr learned my addr 172.21.15.3:0/1428640592 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a2a00700 1 --2- 172.21.15.3:0/1428640592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415cc40 0x7f15a406fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a2000700 1 --2- 172.21.15.3:0/1428640592 >> v2:172.21.15.157:3300/0 conn(0x7f15a406d540 0x7f15a40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 -- 172.21.15.3:0/1428640592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415cc40 msgr2=0x7f15a406fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 --2- 172.21.15.3:0/1428640592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415cc40 0x7f15a406fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 -- 172.21.15.3:0/1428640592 >> v2:172.21.15.157:3300/0 conn(0x7f15a406d540 msgr2=0x7f15a40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 --2- 172.21.15.3:0/1428640592 >> v2:172.21.15.157:3300/0 conn(0x7f15a406d540 0x7f15a40ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 -- 172.21.15.3:0/1428640592 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40ad610 con 0x7f15a415d020 2024-09-13T16:54:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.970+0000 7f15a3400700 1 --2- 172.21.15.3:0/1428640592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415d020 0x7f15a40ace10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f15a406b010 tx=0x7f159c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b078eceee44fb47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a1600700 1 -- 172.21.15.3:0/1428640592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c00e070 con 0x7f15a415d020 2024-09-13T16:54:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a1600700 1 -- 172.21.15.3:0/1428640592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c00b910 con 0x7f15a415d020 2024-09-13T16:54:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a1600700 1 -- 172.21.15.3:0/1428640592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c00bcc0 con 0x7f15a415d020 2024-09-13T16:54:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 -- 172.21.15.3:0/1428640592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415d020 msgr2=0x7f15a40ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1428640592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415d020 0x7f15a40ace10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f15a406b010 tx=0x7f159c00b330 comp rx=0 tx=0).stop 2024-09-13T16:54:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 -- 172.21.15.3:0/1428640592 shutdown_connections 2024-09-13T16:54:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1428640592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415d020 0x7f15a40ace10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1428640592 >> v2:172.21.15.157:3300/0 conn(0x7f15a406d540 0x7f15a40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1428640592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415cc40 0x7f15a406fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 -- 172.21.15.3:0/1428640592 >> 172.21.15.3:0/1428640592 conn(0x7f15a406a2f0 msgr2=0x7f15a406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 -- 172.21.15.3:0/1428640592 shutdown_connections 2024-09-13T16:54:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 -- 172.21.15.3:0/1428640592 wait complete. 2024-09-13T16:54:03.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.974+0000 7f15a3e00700 1 Processor -- start 2024-09-13T16:54:03.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3e00700 1 -- start start 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15a406d540 0x7f15a41543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415cc40 0x7f15a415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415d020 0x7f15a415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4154930 con 0x7f15a415cc40 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a415b970 con 0x7f15a406d540 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a415e470 con 0x7f15a415d020 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415cc40 0x7f15a415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415cc40 0x7f15a415f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38698/0 (socket says 172.21.15.3:38698) 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 -- 172.21.15.3:0/1185941495 learned_addr learned my addr 172.21.15.3:0/1185941495 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3400700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415d020 0x7f15a415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2a00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15a406d540 0x7f15a41543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 -- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415d020 msgr2=0x7f15a415fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415d020 0x7f15a415fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 -- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15a406d540 msgr2=0x7f15a41543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15a406d540 0x7f15a41543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 -- 172.21.15.3:0/1185941495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c009b30 con 0x7f15a415cc40 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a2000700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415cc40 0x7f15a415f1f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f159000e810 tx=0x7f159000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f159aa00700 1 -- 172.21.15.3:0/1185941495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590004230 con 0x7f15a415cc40 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f159aa00700 1 -- 172.21.15.3:0/1185941495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590004390 con 0x7f15a415cc40 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f159aa00700 1 -- 172.21.15.3:0/1185941495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900036c0 con 0x7f15a415cc40 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.978+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a41f7650 con 0x7f15a415cc40 2024-09-13T16:54:03.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.982+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a415e9f0 con 0x7f15a415cc40 2024-09-13T16:54:03.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.982+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a406fc20 con 0x7f15a415cc40 2024-09-13T16:54:03.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.982+0000 7f159aa00700 1 -- 172.21.15.3:0/1185941495 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1590003840 con 0x7f15a415cc40 2024-09-13T16:54:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.986+0000 7f159aa00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1584066760 0x7f1584068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.986+0000 7f159aa00700 1 -- 172.21.15.3:0/1185941495 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f1590083d50 con 0x7f15a415cc40 2024-09-13T16:54:03.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.986+0000 7f15a2a00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1584066760 0x7f1584068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.986+0000 7f15a2a00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1584066760 0x7f1584068c20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1594009bf0 tx=0x7f1594009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:04.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:03.994+0000 7f159aa00700 1 -- 172.21.15.3:0/1185941495 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1590050df0 con 0x7f15a415cc40 2024-09-13T16:54:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.234+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15a4157680 con 0x7f15a415cc40 2024-09-13T16:54:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.238+0000 7f159aa00700 1 -- 172.21.15.3:0/1185941495 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f159000c090 con 0x7f15a415cc40 2024-09-13T16:54:04.244 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:04.244 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:54:04.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1584066760 msgr2=0x7f1584068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1584066760 0x7f1584068c20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1594009bf0 tx=0x7f1594009240 comp rx=0 tx=0).stop 2024-09-13T16:54:04.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415cc40 msgr2=0x7f15a415f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415cc40 0x7f15a415f1f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f159000e810 tx=0x7f159000eb20 comp rx=0 tx=0).stop 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 shutdown_connections 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1584066760 0x7f1584068c20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15a415d020 0x7f15a415fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15a415cc40 0x7f15a415f1f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 --2- 172.21.15.3:0/1185941495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15a406d540 0x7f15a41543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 >> 172.21.15.3:0/1185941495 conn(0x7f15a406a2f0 msgr2=0x7f15a4157ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 shutdown_connections 2024-09-13T16:54:04.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.242+0000 7f15a3e00700 1 -- 172.21.15.3:0/1185941495 wait complete. 2024-09-13T16:54:04.253 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:54:04.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:04.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.442+0000 7fc547200700 1 Processor -- start 2024-09-13T16:54:04.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.442+0000 7fc547200700 1 -- start start 2024-09-13T16:54:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.442+0000 7fc547200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc54806ba80 0x7fc54806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.442+0000 7fc547200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5480ce5d0 con 0x7fc54806be60 2024-09-13T16:54:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc547200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54815af00 con 0x7fc548070dc0 2024-09-13T16:54:04.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc547200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480cd9c0 con 0x7fc54806ba80 2024-09-13T16:54:04.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc54806be60 0x7fc54806c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43670/0 (socket says 172.21.15.3:43670) 2024-09-13T16:54:04.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545400700 1 -- 172.21.15.3:0/507099882 learned_addr learned my addr 172.21.15.3:0/507099882 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:04.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545e00700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc54806ba80 0x7fc54806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102022881 0 0) 0x7fc5480ce5d0 con 0x7fc54806be60 2024-09-13T16:54:04.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc52c0035d0 con 0x7fc54806be60 2024-09-13T16:54:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611160090 0 0) 0x7fc54815af00 con 0x7fc548070dc0 2024-09-13T16:54:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5480ce5d0 con 0x7fc548070dc0 2024-09-13T16:54:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1411856607 0 0) 0x7fc52c0035d0 con 0x7fc54806be60 2024-09-13T16:54:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc54806ba80 msgr2=0x7fc54806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc54806ba80 0x7fc54806d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 >> v1:172.21.15.157:6789/0 conn(0x7fc548070dc0 legacy=0x7fc5481646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480cdca0 con 0x7fc54806be60 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fc534001670 con 0x7fc54806be60 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 >> v1:172.21.15.3:6789/0 conn(0x7fc54806be60 legacy=0x7fc54806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc54806ba80 0x7fc52c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc52c004140 0x7fc52c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004db0 0x7fc52c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480cd9c0 con 0x7fc52c004140 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c00bb10 con 0x7fc52c004db0 2024-09-13T16:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c00bcd0 con 0x7fc54806ba80 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545400700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc52c004140 0x7fc52c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545e00700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc54806ba80 0x7fc52c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc546800700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004db0 0x7fc52c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545400700 1 -- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc54806ba80 msgr2=0x7fc52c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545400700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc54806ba80 0x7fc52c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545400700 1 -- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004db0 msgr2=0x7fc52c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545400700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004db0 0x7fc52c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.446+0000 7fc545400700 1 -- 172.21.15.3:0/507099882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0035d0 con 0x7fc52c004140 2024-09-13T16:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc546800700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004db0 0x7fc52c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc545400700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc52c004140 0x7fc52c004870 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc5480ae2d0 tx=0x7fc52c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e52c6741fb3c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53400ccf0 con 0x7fc52c004140 2024-09-13T16:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc534001af0 con 0x7fc52c004140 2024-09-13T16:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc544a00700 1 -- 172.21.15.3:0/507099882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534005bc0 con 0x7fc52c004140 2024-09-13T16:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc547200700 1 -- 172.21.15.3:0/507099882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc52c004140 msgr2=0x7fc52c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc547200700 1 --2- 172.21.15.3:0/507099882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc52c004140 0x7fc52c004870 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc5480ae2d0 tx=0x7fc52c0058a0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc546800700 1 -- 172.21.15.3:0/507099882 reap_dead start 2024-09-13T16:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc547200700 1 -- 172.21.15.3:0/507099882 shutdown_connections 2024-09-13T16:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc547200700 1 -- 172.21.15.3:0/507099882 >> 172.21.15.3:0/507099882 conn(0x7fc54806a2f0 msgr2=0x7fc54806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc547200700 1 -- 172.21.15.3:0/507099882 shutdown_connections 2024-09-13T16:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.450+0000 7fc547200700 1 -- 172.21.15.3:0/507099882 wait complete. 2024-09-13T16:54:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc547200700 1 Processor -- start 2024-09-13T16:54:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc547200700 1 -- start start 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc547200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004140 0x7fc548148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc547200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54814ab10 0x7fc5480ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc547200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5480aeac0 0x7fc54814b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc547200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480aef40 con 0x7fc54814ab10 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc547200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54814a680 con 0x7fc52c004140 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc547200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc548149a70 con 0x7fc5480aeac0 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc545400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54814ab10 0x7fc5480ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc545400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54814ab10 0x7fc5480ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38726/0 (socket says 172.21.15.3:38726) 2024-09-13T16:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc545400700 1 -- 172.21.15.3:0/456260146 learned_addr learned my addr 172.21.15.3:0/456260146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc545e00700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004140 0x7fc548148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.454+0000 7fc546800700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5480aeac0 0x7fc54814b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc545400700 1 -- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5480aeac0 msgr2=0x7fc54814b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc545400700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5480aeac0 0x7fc54814b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc545400700 1 -- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004140 msgr2=0x7fc548148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc545400700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004140 0x7fc548148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc545400700 1 -- 172.21.15.3:0/456260146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc548149d50 con 0x7fc54814ab10 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc546800700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5480aeac0 0x7fc54814b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc545400700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54814ab10 0x7fc5480ae580 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc534007fd0 tx=0x7fc534001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc53e000700 1 -- 172.21.15.3:0/456260146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534004ca0 con 0x7fc54814ab10 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc53e000700 1 -- 172.21.15.3:0/456260146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc534004e00 con 0x7fc54814ab10 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc53e000700 1 -- 172.21.15.3:0/456260146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340072f0 con 0x7fc54814ab10 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54814a030 con 0x7fc54814ab10 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc548148ed0 con 0x7fc54814ab10 2024-09-13T16:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.458+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548149090 con 0x7fc54814ab10 2024-09-13T16:54:04.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.462+0000 7fc53e000700 1 -- 172.21.15.3:0/456260146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc534019070 con 0x7fc54814ab10 2024-09-13T16:54:04.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.462+0000 7fc53e000700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc528066730 0x7fc528068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.462+0000 7fc53e000700 1 -- 172.21.15.3:0/456260146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fc534084610 con 0x7fc54814ab10 2024-09-13T16:54:04.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.462+0000 7fc545e00700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc528066730 0x7fc528068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.462+0000 7fc545e00700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc528066730 0x7fc528068bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc538014430 tx=0x7fc53800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:04.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.470+0000 7fc53e000700 1 -- 172.21.15.3:0/456260146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc548149090 con 0x7fc54814ab10 2024-09-13T16:54:04.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.698+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc548149090 con 0x7fc54814ab10 2024-09-13T16:54:04.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.698+0000 7fc53e000700 1 -- 172.21.15.3:0/456260146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fc548149090 con 0x7fc54814ab10 2024-09-13T16:54:04.707 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:04.708 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:53:00.113793+0000","modified":"2024-09-13T16:53:12.847855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5983":{"gid":5983,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6835/2114372521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2114372521},{"type":"v1","addr":"172.21.15.3:6835","nonce":2114372521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc528066730 msgr2=0x7fc528068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc528066730 0x7fc528068bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc538014430 tx=0x7fc53800b040 comp rx=0 tx=0).stop 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54814ab10 msgr2=0x7fc5480ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54814ab10 0x7fc5480ae580 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc534007fd0 tx=0x7fc534001980 comp rx=0 tx=0).stop 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 shutdown_connections 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc528066730 0x7fc528068bf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5480aeac0 0x7fc54814b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54814ab10 0x7fc5480ae580 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 --2- 172.21.15.3:0/456260146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c004140 0x7fc548148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 >> 172.21.15.3:0/456260146 conn(0x7fc54806a2f0 msgr2=0x7fc54806d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 shutdown_connections 2024-09-13T16:54:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.702+0000 7fc547200700 1 -- 172.21.15.3:0/456260146 wait complete. 2024-09-13T16:54:04.713 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-13T16:54:04.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:54:04.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd30a00700 1 Processor -- start 2024-09-13T16:54:04.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd30a00700 1 -- start start 2024-09-13T16:54:04.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd30a00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd2c15aaa0 0x7fdd2c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd30a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c0ca290 0x7fdd2c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd30a00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c15ae80 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd30a00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c06af40 con 0x7fdd2c15aaa0 2024-09-13T16:54:04.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd30a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c06b200 con 0x7fdd2c0ca290 2024-09-13T16:54:04.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd2b400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd2c15aaa0 0x7fdd2c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd2aa00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdd2c0ad3d0 0x7fdd2c06a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43674/0 (socket says 172.21.15.3:43674) 2024-09-13T16:54:04.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.906+0000 7fdd2aa00700 1 -- 172.21.15.3:0/1077272972 learned_addr learned my addr 172.21.15.3:0/1077272972 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:04.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2be00700 1 --2- 172.21.15.3:0/1077272972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c0ca290 0x7fdd2c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2a000700 1 -- 172.21.15.3:0/1077272972 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381154815 0 0) 0x7fdd2c15ae80 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2a000700 1 -- 172.21.15.3:0/1077272972 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd100035d0 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2b400700 1 -- 172.21.15.3:0/1077272972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c0ca290 msgr2=0x7fdd2c0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:04.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2b400700 1 --2- 172.21.15.3:0/1077272972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c0ca290 0x7fdd2c0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2b400700 1 -- 172.21.15.3:0/1077272972 >> v1:172.21.15.3:6789/0 conn(0x7fdd2c0ad3d0 legacy=0x7fdd2c06a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2b400700 1 -- 172.21.15.3:0/1077272972 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c06b4e0 con 0x7fdd2c15aaa0 2024-09-13T16:54:04.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2be00700 1 --2- 172.21.15.3:0/1077272972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c0ca290 0x7fdd2c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:04.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2b400700 1 --2- 172.21.15.3:0/1077272972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c15aaa0 0x7fdd2c06a2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009850 tx=0x7fdd1c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7399c9281c655c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2a000700 1 -- 172.21.15.3:0/1077272972 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c011830 con 0x7fdd2c15aaa0 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2a000700 1 -- 172.21.15.3:0/1077272972 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c011990 con 0x7fdd2c15aaa0 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd2a000700 1 -- 172.21.15.3:0/1077272972 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c011d40 con 0x7fdd2c15aaa0 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd30a00700 1 -- 172.21.15.3:0/1077272972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c15aaa0 msgr2=0x7fdd2c06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd30a00700 1 --2- 172.21.15.3:0/1077272972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c15aaa0 0x7fdd2c06a2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009850 tx=0x7fdd1c009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd30a00700 1 -- 172.21.15.3:0/1077272972 shutdown_connections 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd30a00700 1 --2- 172.21.15.3:0/1077272972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c0ca290 0x7fdd2c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd30a00700 1 --2- 172.21.15.3:0/1077272972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c15aaa0 0x7fdd2c06a2f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd30a00700 1 -- 172.21.15.3:0/1077272972 >> 172.21.15.3:0/1077272972 conn(0x7fdd2c0cf010 msgr2=0x7fdd2c0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd30a00700 1 -- 172.21.15.3:0/1077272972 shutdown_connections 2024-09-13T16:54:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.910+0000 7fdd30a00700 1 -- 172.21.15.3:0/1077272972 wait complete. 2024-09-13T16:54:04.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd30a00700 1 Processor -- start 2024-09-13T16:54:04.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd30a00700 1 -- start start 2024-09-13T16:54:04.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd30a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c0ad3d0 0x7fdd2c1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd30a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2c0ca290 0x7fdd2c1e5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd30a00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c15aaa0 0x7fdd2c09bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd30a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c1e4190 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd30a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c1e43e0 con 0x7fdd2c0ca290 2024-09-13T16:54:04.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd30a00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c1e46c0 con 0x7fdd2c15aaa0 2024-09-13T16:54:04.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c0ad3d0 0x7fdd2c1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c0ad3d0 0x7fdd2c1506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38754/0 (socket says 172.21.15.3:38754) 2024-09-13T16:54:04.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2b400700 1 -- 172.21.15.3:0/3315731483 learned_addr learned my addr 172.21.15.3:0/3315731483 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:04.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2be00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c15aaa0 0x7fdd2c09bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2aa00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2c0ca290 0x7fdd2c1e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2b400700 1 -- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c15aaa0 msgr2=0x7fdd2c09bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2b400700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c15aaa0 0x7fdd2c09bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2b400700 1 -- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2c0ca290 msgr2=0x7fdd2c1e5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2b400700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2c0ca290 0x7fdd2c1e5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.914+0000 7fdd2b400700 1 -- 172.21.15.3:0/3315731483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd24009a70 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.918+0000 7fdd2b400700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c0ad3d0 0x7fdd2c1506e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c014040 tx=0x7fdd1c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.918+0000 7fdd23400700 1 -- 172.21.15.3:0/3315731483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0036c0 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.918+0000 7fdd23400700 1 -- 172.21.15.3:0/3315731483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c01e070 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.918+0000 7fdd23400700 1 -- 172.21.15.3:0/3315731483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c003a80 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.918+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c0094b0 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.918+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c1e4c80 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.918+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c06a2f0 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.918+0000 7fdd23400700 1 -- 172.21.15.3:0/3315731483 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd1c003be0 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.922+0000 7fdd23400700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd0c066730 0x7fdd0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:04.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.922+0000 7fdd23400700 1 -- 172.21.15.3:0/3315731483 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fdd1c020070 con 0x7fdd2c0ad3d0 2024-09-13T16:54:04.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.922+0000 7fdd2aa00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd0c066730 0x7fdd0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:04.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.922+0000 7fdd2aa00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd0c066730 0x7fdd0c068bf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c1e5df0 tx=0x7fdd18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:04.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:04.926+0000 7fdd23400700 1 -- 172.21.15.3:0/3315731483 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd1c052960 con 0x7fdd2c0ad3d0 2024-09-13T16:54:05.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.114+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd2c072650 con 0x7fdd2c0ad3d0 2024-09-13T16:54:05.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.118+0000 7fdd23400700 1 -- 172.21.15.3:0/3315731483 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17078 (secure 0 0 0) 0x7fdd1c006ce0 con 0x7fdd2c0ad3d0 2024-09-13T16:54:05.125 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:05.126 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":64,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:53:15.879157+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:52:58.182202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-13T16:52:59.098263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6834/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.157:6836/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.3:6835/291442393":"2024-09-14T16:53:08.782245+0000","172.21.15.3:6834/291442393":"2024-09-14T16:53:08.782245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:54:05.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.118+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd0c066730 msgr2=0x7fdd0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.118+0000 7fdd30a00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd0c066730 0x7fdd0c068bf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c1e5df0 tx=0x7fdd18009040 comp rx=0 tx=0).stop 2024-09-13T16:54:05.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.118+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c0ad3d0 msgr2=0x7fdd2c1506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.118+0000 7fdd30a00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c0ad3d0 0x7fdd2c1506e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c014040 tx=0x7fdd1c004770 comp rx=0 tx=0).stop 2024-09-13T16:54:05.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.122+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 shutdown_connections 2024-09-13T16:54:05.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.122+0000 7fdd30a00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fdd0c066730 0x7fdd0c068bf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.122+0000 7fdd30a00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2c15aaa0 0x7fdd2c09bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.122+0000 7fdd30a00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2c0ca290 0x7fdd2c1e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.122+0000 7fdd30a00700 1 --2- 172.21.15.3:0/3315731483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd2c0ad3d0 0x7fdd2c1506e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.122+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 >> 172.21.15.3:0/3315731483 conn(0x7fdd2c0cf010 msgr2=0x7fdd2c0ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:05.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.122+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 shutdown_connections 2024-09-13T16:54:05.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.122+0000 7fdd30a00700 1 -- 172.21.15.3:0/3315731483 wait complete. 2024-09-13T16:54:05.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T16:54:05.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.326+0000 7fbcfe000700 1 Processor -- start 2024-09-13T16:54:05.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.326+0000 7fbcfe000700 1 -- start start 2024-09-13T16:54:05.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcfe000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbcf815ccd0 0x7fbcf80aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcfe000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b33e0 0x7fbcf806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcfe000700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf8160c50 con 0x7fbcf80b4a60 2024-09-13T16:54:05.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcfe000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf8160f30 con 0x7fbcf815ccd0 2024-09-13T16:54:05.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcfe000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8161210 con 0x7fbcf80b33e0 2024-09-13T16:54:05.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf7400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbcf815ccd0 0x7fbcf80aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf7400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbcf815ccd0 0x7fbcf80aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:59746/0 (socket says 172.21.15.3:59746) 2024-09-13T16:54:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf7400700 1 -- 172.21.15.3:0/191206873 learned_addr learned my addr 172.21.15.3:0/191206873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcefe00700 1 --2- 172.21.15.3:0/191206873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b33e0 0x7fbcf806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf7400700 1 -- 172.21.15.3:0/191206873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b33e0 msgr2=0x7fbcf806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf7400700 1 --2- 172.21.15.3:0/191206873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b33e0 0x7fbcf806a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf7400700 1 -- 172.21.15.3:0/191206873 >> v1:172.21.15.157:6789/0 conn(0x7fbcf80b4a60 legacy=0x7fbcf8070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf7400700 1 -- 172.21.15.3:0/191206873 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf81614f0 con 0x7fbcf815ccd0 2024-09-13T16:54:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf7400700 1 --2- 172.21.15.3:0/191206873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 0x7fbcf80aef30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009910 tx=0x7fbce8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=934b252e07c1abed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.330+0000 7fbcf6a00700 1 -- 172.21.15.3:0/191206873 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8011830 con 0x7fbcf815ccd0 2024-09-13T16:54:05.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcf6a00700 1 -- 172.21.15.3:0/191206873 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce8011990 con 0x7fbcf815ccd0 2024-09-13T16:54:05.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcf6a00700 1 -- 172.21.15.3:0/191206873 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8011d40 con 0x7fbcf815ccd0 2024-09-13T16:54:05.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 -- 172.21.15.3:0/191206873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 msgr2=0x7fbcf80aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 --2- 172.21.15.3:0/191206873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 0x7fbcf80aef30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009910 tx=0x7fbce8009c20 comp rx=0 tx=0).stop 2024-09-13T16:54:05.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 -- 172.21.15.3:0/191206873 shutdown_connections 2024-09-13T16:54:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 --2- 172.21.15.3:0/191206873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b33e0 0x7fbcf806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 --2- 172.21.15.3:0/191206873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 0x7fbcf80aef30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 -- 172.21.15.3:0/191206873 >> 172.21.15.3:0/191206873 conn(0x7fbcf80b1740 msgr2=0x7fbcf80b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 -- 172.21.15.3:0/191206873 shutdown_connections 2024-09-13T16:54:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 -- 172.21.15.3:0/191206873 wait complete. 2024-09-13T16:54:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 Processor -- start 2024-09-13T16:54:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.334+0000 7fbcfe000700 1 -- start start 2024-09-13T16:54:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcfe000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf80b33e0 0x7fbcf81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcfe000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b4a60 0x7fbcf8151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcfe000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 0x7fbcf8153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcfe000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8156b10 con 0x7fbcf80b33e0 2024-09-13T16:54:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcfe000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8151ca0 con 0x7fbcf815ccd0 2024-09-13T16:54:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcfe000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8156d40 con 0x7fbcf80b4a60 2024-09-13T16:54:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf80b33e0 0x7fbcf81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf80b33e0 0x7fbcf81565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38770/0 (socket says 172.21.15.3:38770) 2024-09-13T16:54:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 -- 172.21.15.3:0/693672829 learned_addr learned my addr 172.21.15.3:0/693672829 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcefe00700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b4a60 0x7fbcf8151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7e00700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 0x7fbcf8153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 -- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b4a60 msgr2=0x7fbcf8151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b4a60 0x7fbcf8151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 -- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 msgr2=0x7fbcf8153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 0x7fbcf8153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 -- 172.21.15.3:0/693672829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8009570 con 0x7fbcf80b33e0 2024-09-13T16:54:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcf7400700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf80b33e0 0x7fbcf81565d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8072200 tx=0x7fbce8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcef400700 1 -- 172.21.15.3:0/693672829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8004450 con 0x7fbcf80b33e0 2024-09-13T16:54:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcef400700 1 -- 172.21.15.3:0/693672829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce801f070 con 0x7fbcf80b33e0 2024-09-13T16:54:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf815d230 con 0x7fbcf80b33e0 2024-09-13T16:54:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcef400700 1 -- 172.21.15.3:0/693672829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8003900 con 0x7fbcf80b33e0 2024-09-13T16:54:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.338+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf815d9f0 con 0x7fbcf80b33e0 2024-09-13T16:54:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.342+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf80b2490 con 0x7fbcf80b33e0 2024-09-13T16:54:05.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.342+0000 7fbcef400700 1 -- 172.21.15.3:0/693672829 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbce8003a60 con 0x7fbcf80b33e0 2024-09-13T16:54:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.346+0000 7fbcef400700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbcd4066730 0x7fbcd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.346+0000 7fbcef400700 1 -- 172.21.15.3:0/693672829 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fbce8021070 con 0x7fbcf80b33e0 2024-09-13T16:54:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.346+0000 7fbcefe00700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbcd4066730 0x7fbcd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.346+0000 7fbcefe00700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbcd4066730 0x7fbcd4068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbcf81556e0 tx=0x7fbce0009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.350+0000 7fbcef400700 1 -- 172.21.15.3:0/693672829 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbce8056670 con 0x7fbcf80b33e0 2024-09-13T16:54:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.574+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbcf80aef30 con 0x7fbcf80b33e0 2024-09-13T16:54:05.583 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-13T16:54:05.574+0000 7f9c4ac00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:54:05.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.590+0000 7fbcef400700 1 -- 172.21.15.3:0/693672829 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbce8015090 con 0x7fbcf80b33e0 2024-09-13T16:54:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.590+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbcd4066730 msgr2=0x7fbcd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.590+0000 7fbcfe000700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbcd4066730 0x7fbcd4068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbcf81556e0 tx=0x7fbce0009240 comp rx=0 tx=0).stop 2024-09-13T16:54:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf80b33e0 msgr2=0x7fbcf81565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf80b33e0 0x7fbcf81565d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8072200 tx=0x7fbce8004770 comp rx=0 tx=0).stop 2024-09-13T16:54:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 shutdown_connections 2024-09-13T16:54:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbcd4066730 0x7fbcd4068bf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf815ccd0 0x7fbcf8153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf80b4a60 0x7fbcf8151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 --2- 172.21.15.3:0/693672829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf80b33e0 0x7fbcf81565d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 >> 172.21.15.3:0/693672829 conn(0x7fbcf80b1740 msgr2=0x7fbcf806e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 shutdown_connections 2024-09-13T16:54:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.594+0000 7fbcfe000700 1 -- 172.21.15.3:0/693672829 wait complete. 2024-09-13T16:54:05.602 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T16:54:05.609 INFO:tasks.ceph.mds.c.smithi003.stderr: -2649> 2024-09-13T16:53:13.827+0000 7fd276200700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:54:05.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T16:54:05.662 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-13T16:54:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.794+0000 7fc46e000700 1 Processor -- start 2024-09-13T16:54:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc46e000700 1 -- start start 2024-09-13T16:54:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc46e000700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc468071e60 0x7fc46815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc46e000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4680b2ba0 0x7fc4680ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc46e000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4680cd130 con 0x7fc4680ce470 2024-09-13T16:54:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc46e000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4680cd3d0 con 0x7fc4680b2ba0 2024-09-13T16:54:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc46e000700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4680cd6b0 con 0x7fc468071e60 2024-09-13T16:54:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4680b2ba0 0x7fc4680ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4680b2ba0 0x7fc4680ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38806/0 (socket says 172.21.15.3:38806) 2024-09-13T16:54:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467e00700 1 -- 172.21.15.3:0/427887330 learned_addr learned my addr 172.21.15.3:0/427887330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467400700 1 --2- 172.21.15.3:0/427887330 >> v2:172.21.15.157:3300/0 conn(0x7fc468071e60 0x7fc46815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467e00700 1 -- 172.21.15.3:0/427887330 >> v2:172.21.15.157:3300/0 conn(0x7fc468071e60 msgr2=0x7fc46815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467e00700 1 --2- 172.21.15.3:0/427887330 >> v2:172.21.15.157:3300/0 conn(0x7fc468071e60 0x7fc46815dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467e00700 1 -- 172.21.15.3:0/427887330 >> v1:172.21.15.3:6789/0 conn(0x7fc4680ce470 legacy=0x7fc4680b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467e00700 1 -- 172.21.15.3:0/427887330 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4680cd990 con 0x7fc4680b2ba0 2024-09-13T16:54:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.798+0000 7fc467e00700 1 --2- 172.21.15.3:0/427887330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680b2ba0 0x7fc4680ac8d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc4680cc7c0 tx=0x7fc46000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4457cf72b0c5118 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc466a00700 1 -- 172.21.15.3:0/427887330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46000e070 con 0x7fc4680b2ba0 2024-09-13T16:54:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc466a00700 1 -- 172.21.15.3:0/427887330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46000b9c0 con 0x7fc4680b2ba0 2024-09-13T16:54:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc466a00700 1 -- 172.21.15.3:0/427887330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46000bd70 con 0x7fc4680b2ba0 2024-09-13T16:54:05.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc46e000700 1 -- 172.21.15.3:0/427887330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680b2ba0 msgr2=0x7fc4680ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc46e000700 1 --2- 172.21.15.3:0/427887330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680b2ba0 0x7fc4680ac8d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc4680cc7c0 tx=0x7fc46000b330 comp rx=0 tx=0).stop 2024-09-13T16:54:05.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc46e000700 1 -- 172.21.15.3:0/427887330 shutdown_connections 2024-09-13T16:54:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc46e000700 1 --2- 172.21.15.3:0/427887330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680b2ba0 0x7fc4680ac8d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc46e000700 1 --2- 172.21.15.3:0/427887330 >> v2:172.21.15.157:3300/0 conn(0x7fc468071e60 0x7fc46815dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc46e000700 1 -- 172.21.15.3:0/427887330 >> 172.21.15.3:0/427887330 conn(0x7fc4680b07a0 msgr2=0x7fc4680b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc46e000700 1 -- 172.21.15.3:0/427887330 shutdown_connections 2024-09-13T16:54:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.802+0000 7fc46e000700 1 -- 172.21.15.3:0/427887330 wait complete. 2024-09-13T16:54:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc46e000700 1 Processor -- start 2024-09-13T16:54:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc46e000700 1 -- start start 2024-09-13T16:54:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc46e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc468071e60 0x7fc468153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc46e000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4680b2ba0 0x7fc4681611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc46e000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680ce470 0x7fc46815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc46e000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc468155f30 con 0x7fc4680ce470 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc46e000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc468153580 con 0x7fc468071e60 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc46e000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc468161710 con 0x7fc4680b2ba0 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680ce470 0x7fc46815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680ce470 0x7fc46815ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38812/0 (socket says 172.21.15.3:38812) 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 -- 172.21.15.3:0/198034909 learned_addr learned my addr 172.21.15.3:0/198034909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc45fe00700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4680b2ba0 0x7fc4681611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467400700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc468071e60 0x7fc468153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 -- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4680b2ba0 msgr2=0x7fc4681611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4680b2ba0 0x7fc4681611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 -- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc468071e60 msgr2=0x7fc468153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc468071e60 0x7fc468153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 -- 172.21.15.3:0/198034909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc460009b30 con 0x7fc4680ce470 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc45fe00700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4680b2ba0 0x7fc4681611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467400700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc468071e60 0x7fc468153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.806+0000 7fc467e00700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680ce470 0x7fc46815ebe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc4680746c0 tx=0x7fc460004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.810+0000 7fc45f400700 1 -- 172.21.15.3:0/198034909 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600092e0 con 0x7fc4680ce470 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.810+0000 7fc45f400700 1 -- 172.21.15.3:0/198034909 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46001b430 con 0x7fc4680ce470 2024-09-13T16:54:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.810+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc468156140 con 0x7fc4680ce470 2024-09-13T16:54:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.810+0000 7fc45f400700 1 -- 172.21.15.3:0/198034909 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46001b7e0 con 0x7fc4680ce470 2024-09-13T16:54:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.810+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc468156700 con 0x7fc4680ce470 2024-09-13T16:54:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.810+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46815dd70 con 0x7fc4680ce470 2024-09-13T16:54:05.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.810+0000 7fc45f400700 1 -- 172.21.15.3:0/198034909 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc46001b940 con 0x7fc4680ce470 2024-09-13T16:54:05.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.814+0000 7fc45f400700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc444066730 0x7fc444068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.814+0000 7fc45f400700 1 -- 172.21.15.3:0/198034909 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fc460085a20 con 0x7fc4680ce470 2024-09-13T16:54:05.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.814+0000 7fc467400700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc444066730 0x7fc444068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.814+0000 7fc467400700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc444066730 0x7fc444068bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc468153ae0 tx=0x7fc458008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:05.818+0000 7fc45f400700 1 -- 172.21.15.3:0/198034909 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc460052b00 con 0x7fc4680ce470 2024-09-13T16:54:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.038+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc4680cc9c0 con 0x7fc4680ce470 2024-09-13T16:54:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.594+0000 7fc45f400700 1 -- 172.21.15.3:0/198034909 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc460056120 con 0x7fc4680ce470 2024-09-13T16:54:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc444066730 msgr2=0x7fc444068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc444066730 0x7fc444068bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc468153ae0 tx=0x7fc458008040 comp rx=0 tx=0).stop 2024-09-13T16:54:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680ce470 msgr2=0x7fc46815ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680ce470 0x7fc46815ebe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc4680746c0 tx=0x7fc460004af0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 shutdown_connections 2024-09-13T16:54:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fc444066730 0x7fc444068bf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4680ce470 0x7fc46815ebe0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4680b2ba0 0x7fc4681611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 --2- 172.21.15.3:0/198034909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc468071e60 0x7fc468153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 >> 172.21.15.3:0/198034909 conn(0x7fc4680b07a0 msgr2=0x7fc4680ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 shutdown_connections 2024-09-13T16:54:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.598+0000 7fc46e000700 1 -- 172.21.15.3:0/198034909 wait complete. 2024-09-13T16:54:06.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T16:54:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.850+0000 7f56d1200700 1 Processor -- start 2024-09-13T16:54:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.850+0000 7f56d1200700 1 -- start start 2024-09-13T16:54:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56d1200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56cc160f50 0x7f56cc162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56d1200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f56cc0cb330 0x7f56cc0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56d1200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc1624b0 con 0x7f56cc0b27b0 2024-09-13T16:54:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56d1200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc0cbce0 con 0x7f56cc0cb330 2024-09-13T16:54:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56d1200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc0b19f0 con 0x7f56cc160f50 2024-09-13T16:54:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56c3e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f56cc0cb330 0x7f56cc0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56cb400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56cc160f50 0x7f56cc162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56c3e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f56cc0cb330 0x7f56cc0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38822/0 (socket says 172.21.15.3:38822) 2024-09-13T16:54:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56cbe00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f56cc0b27b0 0x7f56cc0b2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43704/0 (socket says 172.21.15.3:43704) 2024-09-13T16:54:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56c3e00700 1 -- 172.21.15.3:0/3044694585 learned_addr learned my addr 172.21.15.3:0/3044694585 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56c3e00700 1 -- 172.21.15.3:0/3044694585 >> v2:172.21.15.157:3300/0 conn(0x7f56cc160f50 msgr2=0x7f56cc162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56c3e00700 1 --2- 172.21.15.3:0/3044694585 >> v2:172.21.15.157:3300/0 conn(0x7f56cc160f50 0x7f56cc162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56c3e00700 1 -- 172.21.15.3:0/3044694585 >> v1:172.21.15.3:6789/0 conn(0x7f56cc0b27b0 legacy=0x7f56cc0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56caa00700 1 -- 172.21.15.3:0/3044694585 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220856979 0 0) 0x7f56cc1624b0 con 0x7f56cc0b27b0 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56c3e00700 1 -- 172.21.15.3:0/3044694585 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0b1cd0 con 0x7f56cc0cb330 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.854+0000 7f56c3e00700 1 --2- 172.21.15.3:0/3044694585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0cb7a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f56cc06f750 tx=0x7f56b4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bd98c75001640 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56caa00700 1 -- 172.21.15.3:0/3044694585 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4011830 con 0x7f56cc0cb330 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56caa00700 1 -- 172.21.15.3:0/3044694585 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b4011990 con 0x7f56cc0cb330 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56caa00700 1 -- 172.21.15.3:0/3044694585 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4011d20 con 0x7f56cc0cb330 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 -- 172.21.15.3:0/3044694585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 msgr2=0x7f56cc0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 --2- 172.21.15.3:0/3044694585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0cb7a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f56cc06f750 tx=0x7f56b4009bf0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 -- 172.21.15.3:0/3044694585 shutdown_connections 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 --2- 172.21.15.3:0/3044694585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0cb7a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 --2- 172.21.15.3:0/3044694585 >> v2:172.21.15.157:3300/0 conn(0x7f56cc160f50 0x7f56cc162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 -- 172.21.15.3:0/3044694585 >> 172.21.15.3:0/3044694585 conn(0x7f56cc06a2f0 msgr2=0x7f56cc06a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 -- 172.21.15.3:0/3044694585 shutdown_connections 2024-09-13T16:54:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 -- 172.21.15.3:0/3044694585 wait complete. 2024-09-13T16:54:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.858+0000 7f56d1200700 1 Processor -- start 2024-09-13T16:54:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56d1200700 1 -- start start 2024-09-13T16:54:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56d1200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56cc0b27b0 0x7f56cc157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56d1200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56d1200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56cc160f50 0x7f56cc0c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56d1200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc15a030 con 0x7f56cc0cb330 2024-09-13T16:54:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56d1200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc158070 con 0x7f56cc0b27b0 2024-09-13T16:54:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56d1200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1593b0 con 0x7f56cc160f50 2024-09-13T16:54:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38834/0 (socket says 172.21.15.3:38834) 2024-09-13T16:54:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 -- 172.21.15.3:0/3902265799 learned_addr learned my addr 172.21.15.3:0/3902265799 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56cb400700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56cc0b27b0 0x7f56cc157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56cbe00700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56cc160f50 0x7f56cc0c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 -- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56cc160f50 msgr2=0x7f56cc0c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56cc160f50 0x7f56cc0c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 -- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56cc0b27b0 msgr2=0x7f56cc157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:06.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56cc0b27b0 0x7f56cc157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 -- 172.21.15.3:0/3902265799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4009570 con 0x7f56cc0cb330 2024-09-13T16:54:06.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.862+0000 7f56c3e00700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0c5370 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009f40 tx=0x7f56b4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:06.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.866+0000 7f56c3400700 1 -- 172.21.15.3:0/3902265799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4004380 con 0x7f56cc0cb330 2024-09-13T16:54:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.866+0000 7f56c3400700 1 -- 172.21.15.3:0/3902265799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b401f070 con 0x7f56cc0cb330 2024-09-13T16:54:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.866+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc159690 con 0x7f56cc0cb330 2024-09-13T16:54:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.866+0000 7f56c3400700 1 -- 172.21.15.3:0/3902265799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40036c0 con 0x7f56cc0cb330 2024-09-13T16:54:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.866+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc159c50 con 0x7f56cc0cb330 2024-09-13T16:54:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.866+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc15a170 con 0x7f56cc0cb330 2024-09-13T16:54:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.866+0000 7f56c3400700 1 -- 172.21.15.3:0/3902265799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f56b4021030 con 0x7f56cc0cb330 2024-09-13T16:54:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.870+0000 7f56c3400700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f56a8066760 0x7f56a8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.870+0000 7f56c3400700 1 -- 172.21.15.3:0/3902265799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f56b4085760 con 0x7f56cc0cb330 2024-09-13T16:54:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.870+0000 7f56cb400700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f56a8066760 0x7f56a8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.870+0000 7f56cb400700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f56a8066760 0x7f56a8068c20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009b50 tx=0x7f56bc009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:06.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:06.874+0000 7f56c3400700 1 -- 172.21.15.3:0/3902265799 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f56cc15a170 con 0x7f56cc0cb330 2024-09-13T16:54:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.058+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f56cc15a170 con 0x7f56cc0cb330 2024-09-13T16:54:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.606+0000 7f56c3400700 1 -- 172.21.15.3:0/3902265799 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) v1 ==== 181+0+0 (secure 0 0 0) 0x7f56cc15a170 con 0x7f56cc0cb330 2024-09-13T16:54:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f56a8066760 msgr2=0x7f56a8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f56a8066760 0x7f56a8068c20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009b50 tx=0x7f56bc009240 comp rx=0 tx=0).stop 2024-09-13T16:54:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 msgr2=0x7f56cc0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0c5370 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009f40 tx=0x7f56b4004770 comp rx=0 tx=0).stop 2024-09-13T16:54:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 shutdown_connections 2024-09-13T16:54:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f56a8066760 0x7f56a8068c20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56cc160f50 0x7f56cc0c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56cc0cb330 0x7f56cc0c5370 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 --2- 172.21.15.3:0/3902265799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56cc0b27b0 0x7f56cc157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 >> 172.21.15.3:0/3902265799 conn(0x7f56cc06a2f0 msgr2=0x7f56cc163300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 shutdown_connections 2024-09-13T16:54:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.610+0000 7f56d1200700 1 -- 172.21.15.3:0/3902265799 wait complete. 2024-09-13T16:54:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-13T16:54:07.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T16:54:07.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53e00700 1 Processor -- start 2024-09-13T16:54:07.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53e00700 1 -- start start 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf54153c30 0x7fbf5415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf5406ae60 0x7fbf54160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf5406a2f0 0x7fbf5406a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf54160a70 con 0x7fbf5406a2f0 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5406acc0 con 0x7fbf5406ae60 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54160700 con 0x7fbf54153c30 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf5406a2f0 0x7fbf5406a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf5406a2f0 0x7fbf5406a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38850/0 (socket says 172.21.15.3:38850) 2024-09-13T16:54:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf53400700 1 -- 172.21.15.3:0/1447483728 learned_addr learned my addr 172.21.15.3:0/1447483728 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:07.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf52000700 1 --2- 172.21.15.3:0/1447483728 >> v2:172.21.15.157:3300/0 conn(0x7fbf5406ae60 0x7fbf54160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:07.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.818+0000 7fbf52a00700 1 --2- 172.21.15.3:0/1447483728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf54153c30 0x7fbf5415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:07.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53400700 1 -- 172.21.15.3:0/1447483728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf54153c30 msgr2=0x7fbf5415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53400700 1 --2- 172.21.15.3:0/1447483728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf54153c30 0x7fbf5415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53400700 1 -- 172.21.15.3:0/1447483728 >> v2:172.21.15.157:3300/0 conn(0x7fbf5406ae60 msgr2=0x7fbf54160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53400700 1 --2- 172.21.15.3:0/1447483728 >> v2:172.21.15.157:3300/0 conn(0x7fbf5406ae60 0x7fbf54160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53400700 1 -- 172.21.15.3:0/1447483728 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5415d910 con 0x7fbf5406a2f0 2024-09-13T16:54:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53400700 1 --2- 172.21.15.3:0/1447483728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406a2f0 0x7fbf5406a780 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbf54073340 tx=0x7fbf4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f218f144bf85f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf51600700 1 -- 172.21.15.3:0/1447483728 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00e070 con 0x7fbf5406a2f0 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf51600700 1 -- 172.21.15.3:0/1447483728 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c00b9c0 con 0x7fbf5406a2f0 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf51600700 1 -- 172.21.15.3:0/1447483728 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00bd70 con 0x7fbf5406a2f0 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 -- 172.21.15.3:0/1447483728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406a2f0 msgr2=0x7fbf5406a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 --2- 172.21.15.3:0/1447483728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406a2f0 0x7fbf5406a780 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbf54073340 tx=0x7fbf4c00b330 comp rx=0 tx=0).stop 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 -- 172.21.15.3:0/1447483728 shutdown_connections 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 --2- 172.21.15.3:0/1447483728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406a2f0 0x7fbf5406a780 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 --2- 172.21.15.3:0/1447483728 >> v2:172.21.15.157:3300/0 conn(0x7fbf5406ae60 0x7fbf54160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 --2- 172.21.15.3:0/1447483728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf54153c30 0x7fbf5415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 -- 172.21.15.3:0/1447483728 >> 172.21.15.3:0/1447483728 conn(0x7fbf540b1580 msgr2=0x7fbf540b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 -- 172.21.15.3:0/1447483728 shutdown_connections 2024-09-13T16:54:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.822+0000 7fbf53e00700 1 -- 172.21.15.3:0/1447483728 wait complete. 2024-09-13T16:54:07.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53e00700 1 Processor -- start 2024-09-13T16:54:07.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53e00700 1 -- start start 2024-09-13T16:54:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5406a2f0 0x7fbf54151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406ae60 0x7fbf54150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf54153c30 0x7fbf541556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54155c00 con 0x7fbf5406ae60 2024-09-13T16:54:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54150a00 con 0x7fbf54153c30 2024-09-13T16:54:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54150ce0 con 0x7fbf5406a2f0 2024-09-13T16:54:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf52000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406ae60 0x7fbf54150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf52000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406ae60 0x7fbf54150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38866/0 (socket says 172.21.15.3:38866) 2024-09-13T16:54:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf52000700 1 -- 172.21.15.3:0/2863514844 learned_addr learned my addr 172.21.15.3:0/2863514844 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf52a00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5406a2f0 0x7fbf54151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.826+0000 7fbf53400700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf54153c30 0x7fbf541556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf52000700 1 -- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5406a2f0 msgr2=0x7fbf54151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf52000700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5406a2f0 0x7fbf54151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf52000700 1 -- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf54153c30 msgr2=0x7fbf541556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf52000700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf54153c30 0x7fbf541556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf52000700 1 -- 172.21.15.3:0/2863514844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c009b30 con 0x7fbf5406ae60 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf52000700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406ae60 0x7fbf54150340 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000d800 tx=0x7fbf4000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf4aa00700 1 -- 172.21.15.3:0/2863514844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40004240 con 0x7fbf5406ae60 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf4aa00700 1 -- 172.21.15.3:0/2863514844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf400043a0 con 0x7fbf5406ae60 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf54150fc0 con 0x7fbf5406ae60 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf4aa00700 1 -- 172.21.15.3:0/2863514844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400036c0 con 0x7fbf5406ae60 2024-09-13T16:54:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf54151580 con 0x7fbf5406ae60 2024-09-13T16:54:07.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf540677f0 con 0x7fbf5406ae60 2024-09-13T16:54:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.830+0000 7fbf4aa00700 1 -- 172.21.15.3:0/2863514844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf40003820 con 0x7fbf5406ae60 2024-09-13T16:54:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.834+0000 7fbf4aa00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf34066730 0x7fbf34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.834+0000 7fbf4aa00700 1 -- 172.21.15.3:0/2863514844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5907+0+0 (secure 0 0 0) 0x7fbf40083ad0 con 0x7fbf5406ae60 2024-09-13T16:54:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.834+0000 7fbf52a00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf34066730 0x7fbf34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.834+0000 7fbf52a00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf34066730 0x7fbf34068bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf5415eb50 tx=0x7fbf44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:07.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:07.842+0000 7fbf4aa00700 1 -- 172.21.15.3:0/2863514844 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf40050d80 con 0x7fbf5406ae60 2024-09-13T16:54:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.022+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbf5406a780 con 0x7fbf5406ae60 2024-09-13T16:54:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.610+0000 7fbf4aa00700 1 -- 172.21.15.3:0/2863514844 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbf40019020 con 0x7fbf5406ae60 2024-09-13T16:54:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.614+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf34066730 msgr2=0x7fbf34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.614+0000 7fbf53e00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf34066730 0x7fbf34068bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf5415eb50 tx=0x7fbf44008040 comp rx=0 tx=0).stop 2024-09-13T16:54:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.614+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406ae60 msgr2=0x7fbf54150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.614+0000 7fbf53e00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406ae60 0x7fbf54150340 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000d800 tx=0x7fbf4000db10 comp rx=0 tx=0).stop 2024-09-13T16:54:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.618+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 shutdown_connections 2024-09-13T16:54:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.618+0000 7fbf53e00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fbf34066730 0x7fbf34068bf0 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf5415eb50 tx=0x7fbf44008040 comp rx=0 tx=0).stop 2024-09-13T16:54:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.618+0000 7fbf53e00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf54153c30 0x7fbf541556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.618+0000 7fbf53e00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5406ae60 0x7fbf54150340 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.618+0000 7fbf53e00700 1 --2- 172.21.15.3:0/2863514844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5406a2f0 0x7fbf54151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.618+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 >> 172.21.15.3:0/2863514844 conn(0x7fbf540b1580 msgr2=0x7fbf540ad270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:08.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.618+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 shutdown_connections 2024-09-13T16:54:08.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.618+0000 7fbf53e00700 1 -- 172.21.15.3:0/2863514844 wait complete. 2024-09-13T16:54:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-13T16:54:08.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:54:08.653 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:54:08.718 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-09-13T16:54:08.718 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-09-13T16:54:08.718 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:54:08.718 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-09-13T16:54:08.718 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.719 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:54:08.720 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2024-09-13T16:54:08.721 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-09-13T16:54:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-09-13T16:54:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:54:08.724 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-09-13T16:54:08.725 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.726 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on pool no app = False 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-09-13T16:54:08.727 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-09-13T16:54:08.728 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-09-13T16:54:08.729 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.730 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.731 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:54:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 61456116-a42d-4065-9bf5-a1daf0be594e 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-13T16:54:08.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-13T16:54:08.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-13T16:54:08.735 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:54:08.736 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-13T16:54:08.737 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.738 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-13T16:54:08.739 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-13T16:54:08.740 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-13T16:54:08.741 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-13T16:54:08.741 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-13T16:54:08.741 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-13T16:54:08.741 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-13T16:54:08.741 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-13T16:54:08.741 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-13T16:54:08.741 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-13T16:54:08.741 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-13T16:54:08.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T16:54:08.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.910+0000 7fe7d0e00700 1 Processor -- start 2024-09-13T16:54:08.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.910+0000 7fe7d0e00700 1 -- start start 2024-09-13T16:54:08.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7d0e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7cc06f970 0x7fe7cc06fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7d0e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0ce620 con 0x7fe7cc0cae70 2024-09-13T16:54:08.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7d0e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0ce900 con 0x7fe7cc15ccc0 2024-09-13T16:54:08.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7d0e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc070320 con 0x7fe7cc06f970 2024-09-13T16:54:08.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7ca800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe7cc0cae70 0x7fe7cc0cb2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43710/0 (socket says 172.21.15.3:43710) 2024-09-13T16:54:08.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7ca800700 1 -- 172.21.15.3:0/1280841565 learned_addr learned my addr 172.21.15.3:0/1280841565 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7c9400700 1 --2- 172.21.15.3:0/1280841565 >> v2:172.21.15.157:3300/0 conn(0x7fe7cc06f970 0x7fe7cc06fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7c9400700 1 -- 172.21.15.3:0/1280841565 >> v1:172.21.15.157:6789/0 conn(0x7fe7cc15ccc0 legacy=0x7fe7cc0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7c9400700 1 -- 172.21.15.3:0/1280841565 >> v1:172.21.15.3:6789/0 conn(0x7fe7cc0cae70 legacy=0x7fe7cc0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7c9400700 1 -- 172.21.15.3:0/1280841565 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc0cebe0 con 0x7fe7cc06f970 2024-09-13T16:54:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.914+0000 7fe7c8a00700 1 -- 172.21.15.3:0/1280841565 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878421706 0 0) 0x7fe7cc0ce620 con 0x7fe7cc0cae70 2024-09-13T16:54:08.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7c9400700 1 --2- 172.21.15.3:0/1280841565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 0x7fe7cc06fde0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009850 tx=0x7fe7b8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=247d06588663c7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:08.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7c8a00700 1 -- 172.21.15.3:0/1280841565 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8011830 con 0x7fe7cc06f970 2024-09-13T16:54:08.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7c8a00700 1 -- 172.21.15.3:0/1280841565 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8011990 con 0x7fe7cc06f970 2024-09-13T16:54:08.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7c8a00700 1 -- 172.21.15.3:0/1280841565 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8011d20 con 0x7fe7cc06f970 2024-09-13T16:54:08.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7d0e00700 1 -- 172.21.15.3:0/1280841565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 msgr2=0x7fe7cc06fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7d0e00700 1 --2- 172.21.15.3:0/1280841565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 0x7fe7cc06fde0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009850 tx=0x7fe7b8009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:08.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7d0e00700 1 -- 172.21.15.3:0/1280841565 shutdown_connections 2024-09-13T16:54:08.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7d0e00700 1 --2- 172.21.15.3:0/1280841565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 0x7fe7cc06fde0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7d0e00700 1 -- 172.21.15.3:0/1280841565 >> 172.21.15.3:0/1280841565 conn(0x7fe7cc0b2b00 msgr2=0x7fe7cc0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:08.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7d0e00700 1 -- 172.21.15.3:0/1280841565 shutdown_connections 2024-09-13T16:54:08.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.918+0000 7fe7d0e00700 1 -- 172.21.15.3:0/1280841565 wait complete. 2024-09-13T16:54:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7d0e00700 1 Processor -- start 2024-09-13T16:54:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7d0e00700 1 -- start start 2024-09-13T16:54:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7d0e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 0x7fe7cc154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7d0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc0cae70 0x7fe7cc143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7d0e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7cc15ccc0 0x7fe7cc1f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7d0e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc1579c0 con 0x7fe7cc0cae70 2024-09-13T16:54:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7d0e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc1443e0 con 0x7fe7cc06f970 2024-09-13T16:54:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7d0e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc157400 con 0x7fe7cc15ccc0 2024-09-13T16:54:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc0cae70 0x7fe7cc143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc0cae70 0x7fe7cc143ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38882/0 (socket says 172.21.15.3:38882) 2024-09-13T16:54:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9400700 1 -- 172.21.15.3:0/147179435 learned_addr learned my addr 172.21.15.3:0/147179435 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 0x7fe7cc154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7ca800700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7cc15ccc0 0x7fe7cc1f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9400700 1 -- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7cc15ccc0 msgr2=0x7fe7cc1f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9400700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7cc15ccc0 0x7fe7cc1f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9400700 1 -- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 msgr2=0x7fe7cc154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9400700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 0x7fe7cc154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.922+0000 7fe7c9400700 1 -- 172.21.15.3:0/147179435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80094b0 con 0x7fe7cc0cae70 2024-09-13T16:54:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.926+0000 7fe7c9400700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc0cae70 0x7fe7cc143ea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80090b0 tx=0x7fe7b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.926+0000 7fe7c2000700 1 -- 172.21.15.3:0/147179435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800ba70 con 0x7fe7cc0cae70 2024-09-13T16:54:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.926+0000 7fe7c2000700 1 -- 172.21.15.3:0/147179435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b801f070 con 0x7fe7cc0cae70 2024-09-13T16:54:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.926+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc1576e0 con 0x7fe7cc0cae70 2024-09-13T16:54:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.926+0000 7fe7c2000700 1 -- 172.21.15.3:0/147179435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800bdf0 con 0x7fe7cc0cae70 2024-09-13T16:54:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.926+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc1f6250 con 0x7fe7cc0cae70 2024-09-13T16:54:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.926+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc0ce1f0 con 0x7fe7cc0cae70 2024-09-13T16:54:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.930+0000 7fe7c2000700 1 -- 172.21.15.3:0/147179435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7b801a430 con 0x7fe7cc0cae70 2024-09-13T16:54:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.930+0000 7fe7c2000700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe7ac066730 0x7fe7ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.930+0000 7fe7c2000700 1 -- 172.21.15.3:0/147179435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fe7b8021070 con 0x7fe7cc0cae70 2024-09-13T16:54:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.930+0000 7fe7c9e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe7ac066730 0x7fe7ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.930+0000 7fe7c9e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe7ac066730 0x7fe7ac068bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc000f60 tx=0x7fe7bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:08.938+0000 7fe7c2000700 1 -- 172.21.15.3:0/147179435 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe7b8052c60 con 0x7fe7cc0cae70 2024-09-13T16:54:09.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.122+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe7cc0b3080 con 0x7fe7cc0cae70 2024-09-13T16:54:09.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.130+0000 7fe7c2000700 1 -- 172.21.15.3:0/147179435 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7b8056280 con 0x7fe7cc0cae70 2024-09-13T16:54:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe7ac066730 msgr2=0x7fe7ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe7ac066730 0x7fe7ac068bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc000f60 tx=0x7fe7bc009040 comp rx=0 tx=0).stop 2024-09-13T16:54:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc0cae70 msgr2=0x7fe7cc143ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc0cae70 0x7fe7cc143ea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80090b0 tx=0x7fe7b8004770 comp rx=0 tx=0).stop 2024-09-13T16:54:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 shutdown_connections 2024-09-13T16:54:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fe7ac066730 0x7fe7ac068bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7cc15ccc0 0x7fe7cc1f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc0cae70 0x7fe7cc143ea0 secure :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80090b0 tx=0x7fe7b8004770 comp rx=0 tx=0).stop 2024-09-13T16:54:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 --2- 172.21.15.3:0/147179435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7cc06f970 0x7fe7cc154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 >> 172.21.15.3:0/147179435 conn(0x7fe7cc0b2b00 msgr2=0x7fe7cc0cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 shutdown_connections 2024-09-13T16:54:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.134+0000 7fe7d0e00700 1 -- 172.21.15.3:0/147179435 wait complete. 2024-09-13T16:54:09.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters' 2024-09-13T16:54:09.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.338+0000 7fd0d2800700 1 Processor -- start 2024-09-13T16:54:09.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0d2800700 1 -- start start 2024-09-13T16:54:09.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0d2800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0cc071de0 0x7fd0cc1462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:09.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0d2800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc146c80 con 0x7fd0cc0721c0 2024-09-13T16:54:09.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0d2800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0b4b40 con 0x7fd0cc146810 2024-09-13T16:54:09.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0d2800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b3aa0 con 0x7fd0cc071de0 2024-09-13T16:54:09.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0c3e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd0cc0721c0 0x7fd0cc160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43712/0 (socket says 172.21.15.3:43712) 2024-09-13T16:54:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0c3e00700 1 -- 172.21.15.3:0/3602918344 learned_addr learned my addr 172.21.15.3:0/3602918344 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0cb400700 1 --2- 172.21.15.3:0/3602918344 >> v2:172.21.15.3:3300/0 conn(0x7fd0cc071de0 0x7fd0cc1462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0cb400700 1 -- 172.21.15.3:0/3602918344 >> v1:172.21.15.157:6789/0 conn(0x7fd0cc146810 legacy=0x7fd0cc06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0cb400700 1 -- 172.21.15.3:0/3602918344 >> v1:172.21.15.3:6789/0 conn(0x7fd0cc0721c0 legacy=0x7fd0cc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0cb400700 1 -- 172.21.15.3:0/3602918344 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0b3d80 con 0x7fd0cc071de0 2024-09-13T16:54:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.342+0000 7fd0caa00700 1 -- 172.21.15.3:0/3602918344 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005231113 0 0) 0x7fd0b4001110 con 0x7fd0cc0721c0 2024-09-13T16:54:09.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0cb400700 1 --2- 172.21.15.3:0/3602918344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc071de0 0x7fd0cc1462d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009850 tx=0x7fd0bc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bd51e2e7a43304 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:09.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0caa00700 1 -- 172.21.15.3:0/3602918344 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc011830 con 0x7fd0cc071de0 2024-09-13T16:54:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0caa00700 1 -- 172.21.15.3:0/3602918344 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc011990 con 0x7fd0cc071de0 2024-09-13T16:54:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0caa00700 1 -- 172.21.15.3:0/3602918344 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc011d20 con 0x7fd0cc071de0 2024-09-13T16:54:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0d2800700 1 -- 172.21.15.3:0/3602918344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc071de0 msgr2=0x7fd0cc1462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0d2800700 1 --2- 172.21.15.3:0/3602918344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc071de0 0x7fd0cc1462d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009850 tx=0x7fd0bc009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0d2800700 1 -- 172.21.15.3:0/3602918344 shutdown_connections 2024-09-13T16:54:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0d2800700 1 --2- 172.21.15.3:0/3602918344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc071de0 0x7fd0cc1462d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0d2800700 1 -- 172.21.15.3:0/3602918344 >> 172.21.15.3:0/3602918344 conn(0x7fd0cc071050 msgr2=0x7fd0cc071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0d2800700 1 -- 172.21.15.3:0/3602918344 shutdown_connections 2024-09-13T16:54:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.346+0000 7fd0d2800700 1 -- 172.21.15.3:0/3602918344 wait complete. 2024-09-13T16:54:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0d2800700 1 Processor -- start 2024-09-13T16:54:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0d2800700 1 -- start start 2024-09-13T16:54:09.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0d2800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0cc071de0 0x7fd0cc0b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:09.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0d2800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0721c0 0x7fd0cc159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:09.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0d2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0cc146810 0x7fd0cc159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0d2800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc15bf20 con 0x7fd0cc0721c0 2024-09-13T16:54:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0d2800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc158c90 con 0x7fd0cc146810 2024-09-13T16:54:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0d2800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc15c170 con 0x7fd0cc071de0 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0c3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0721c0 0x7fd0cc159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0c3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0721c0 0x7fd0cc159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38892/0 (socket says 172.21.15.3:38892) 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0c3e00700 1 -- 172.21.15.3:0/1919996339 learned_addr learned my addr 172.21.15.3:0/1919996339 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0cb400700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0cc071de0 0x7fd0cc0b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.350+0000 7fd0cbe00700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0cc146810 0x7fd0cc159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3e00700 1 -- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0cc071de0 msgr2=0x7fd0cc0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3e00700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0cc071de0 0x7fd0cc0b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3e00700 1 -- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0cc146810 msgr2=0x7fd0cc159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3e00700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0cc146810 0x7fd0cc159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3e00700 1 -- 172.21.15.3:0/1919996339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc0094b0 con 0x7fd0cc0721c0 2024-09-13T16:54:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3e00700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0721c0 0x7fd0cc159400 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c550 tx=0x7fd0b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3400700 1 -- 172.21.15.3:0/1919996339 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4004ca0 con 0x7fd0cc0721c0 2024-09-13T16:54:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3400700 1 -- 172.21.15.3:0/1919996339 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4004e00 con 0x7fd0cc0721c0 2024-09-13T16:54:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc15c450 con 0x7fd0cc0721c0 2024-09-13T16:54:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0c3400700 1 -- 172.21.15.3:0/1919996339 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4007310 con 0x7fd0cc0721c0 2024-09-13T16:54:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc137930 con 0x7fd0cc0721c0 2024-09-13T16:54:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.354+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc15c970 con 0x7fd0cc0721c0 2024-09-13T16:54:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.358+0000 7fd0c3400700 1 -- 172.21.15.3:0/1919996339 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0b4007470 con 0x7fd0cc0721c0 2024-09-13T16:54:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.358+0000 7fd0c3400700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd0a8066730 0x7fd0a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.358+0000 7fd0c3400700 1 -- 172.21.15.3:0/1919996339 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd0b4083b20 con 0x7fd0cc0721c0 2024-09-13T16:54:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.358+0000 7fd0cb400700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd0a8066730 0x7fd0a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.362+0000 7fd0cb400700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd0a8066730 0x7fd0a8068bf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0090b0 tx=0x7fd0bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:09.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.366+0000 7fd0c3400700 1 -- 172.21.15.3:0/1919996339 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd0b4050f90 con 0x7fd0cc0721c0 2024-09-13T16:54:09.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:09.630+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]} v 0) v1 -- 0x7fd0cc146c80 con 0x7fd0cc0721c0 2024-09-13T16:54:10.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.630+0000 7fd0c3400700 1 -- 172.21.15.3:0/1919996339 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]}]=0 v205) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd0cc146c80 con 0x7fd0cc0721c0 2024-09-13T16:54:10.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd0a8066730 msgr2=0x7fd0a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd0a8066730 0x7fd0a8068bf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0090b0 tx=0x7fd0bc008030 comp rx=0 tx=0).stop 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0721c0 msgr2=0x7fd0cc159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0721c0 0x7fd0cc159400 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c550 tx=0x7fd0b400c860 comp rx=0 tx=0).stop 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 shutdown_connections 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd0a8066730 0x7fd0a8068bf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0cc146810 0x7fd0cc159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0721c0 0x7fd0cc159400 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 --2- 172.21.15.3:0/1919996339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0cc071de0 0x7fd0cc0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 >> 172.21.15.3:0/1919996339 conn(0x7fd0cc071050 msgr2=0x7fd0cc15e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 shutdown_connections 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:10.634+0000 7fd0d2800700 1 -- 172.21.15.3:0/1919996339 wait complete. 2024-09-13T16:54:10.669 INFO:tasks.cephfs_test_runner:Opening a file should increment omap_total_updates by 1. ... ok 2024-09-13T16:54:10.670 INFO:tasks.cephfs_test_runner: 2024-09-13T16:54:10.670 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-13T16:54:10.670 INFO:tasks.cephfs_test_runner:Ran 2 tests in 160.379s 2024-09-13T16:54:10.670 INFO:tasks.cephfs_test_runner: 2024-09-13T16:54:10.670 INFO:tasks.cephfs_test_runner:OK 2024-09-13T16:54:10.670 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-13T16:54:10.679 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-13T16:54:10.750 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-13T16:54:10.750 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:54:10.751 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:54:10.770 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349932k,nr_inodes=4087483,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4619 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T16:54:10.770 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-13T16:54:10.770 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:54:10.828 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349932k,nr_inodes=4087483,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4619 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T16:54:10.828 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:54:10.828 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:54:10.847 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346844k,nr_inodes=4086711,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T16:54:10.847 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T16:54:10.847 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:54:10.906 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346844k,nr_inodes=4086711,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T16:54:10.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:10.907 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:54:10.928 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T16:54:10.928 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T16:54:10.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:10.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T16:54:11.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:11.006 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T16:54:11.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:11.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-13T16:54:11.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:11.601 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-13T16:54:11.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:11.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-13T16:54:12.012 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-13T16:54:12.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:54:12.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:12.015 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-13T16:54:12.037 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T16:54:12.037 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-13T16:54:12.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:12.038 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T16:54:12.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:12.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-13T16:54:12.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:12.631 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-09-13T16:54:12.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:12.782 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-13T16:54:12.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:54:12.798 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-09-13T16:54:13.180 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-09-13T16:54:13.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:54:13.183 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T16:54:13.194 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T16:54:13.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:54:13.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.378+0000 7f5a00e00700 1 Processor -- start 2024-09-13T16:54:13.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f5a00e00700 1 -- start start 2024-09-13T16:54:13.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f5a00e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc0adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f5a00e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59fc070080 0x7f59fc0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f5a00e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc06a390 con 0x7f59fc15cfa0 2024-09-13T16:54:13.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f5a00e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc06a670 con 0x7f59fc070080 2024-09-13T16:54:13.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f5a00e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc06a950 con 0x7f59fc15cbc0 2024-09-13T16:54:13.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59fb400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc0adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59faa00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f59fc15cfa0 0x7f59fc0ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:34448/0 (socket says 172.21.15.3:34448) 2024-09-13T16:54:13.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59faa00700 1 -- 172.21.15.3:0/3645081847 learned_addr learned my addr 172.21.15.3:0/3645081847 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:13.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59fbe00700 1 --2- 172.21.15.3:0/3645081847 >> v2:172.21.15.157:3300/0 conn(0x7f59fc070080 0x7f59fc0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59fa000700 1 -- 172.21.15.3:0/3645081847 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479321129 0 0) 0x7f59fc06a390 con 0x7f59fc15cfa0 2024-09-13T16:54:13.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59fa000700 1 -- 172.21.15.3:0/3645081847 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59e00035d0 con 0x7f59fc15cfa0 2024-09-13T16:54:13.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59fb400700 1 -- 172.21.15.3:0/3645081847 >> v2:172.21.15.157:3300/0 conn(0x7f59fc070080 msgr2=0x7f59fc0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:13.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59fb400700 1 --2- 172.21.15.3:0/3645081847 >> v2:172.21.15.157:3300/0 conn(0x7f59fc070080 0x7f59fc0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59fb400700 1 -- 172.21.15.3:0/3645081847 >> v1:172.21.15.157:6789/0 conn(0x7f59fc15cfa0 legacy=0x7f59fc0ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.382+0000 7f59fb400700 1 -- 172.21.15.3:0/3645081847 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc06ac30 con 0x7f59fc15cbc0 2024-09-13T16:54:13.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f59fb400700 1 --2- 172.21.15.3:0/3645081847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc0adad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009850 tx=0x7f59ec009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52220fa72bdfed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f59fa000700 1 -- 172.21.15.3:0/3645081847 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec011830 con 0x7f59fc15cbc0 2024-09-13T16:54:13.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f59fa000700 1 -- 172.21.15.3:0/3645081847 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec011990 con 0x7f59fc15cbc0 2024-09-13T16:54:13.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f59fa000700 1 -- 172.21.15.3:0/3645081847 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec011d00 con 0x7f59fc15cbc0 2024-09-13T16:54:13.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f5a00e00700 1 -- 172.21.15.3:0/3645081847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 msgr2=0x7f59fc0adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f5a00e00700 1 --2- 172.21.15.3:0/3645081847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc0adad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009850 tx=0x7f59ec009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:13.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f5a00e00700 1 -- 172.21.15.3:0/3645081847 shutdown_connections 2024-09-13T16:54:13.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f5a00e00700 1 --2- 172.21.15.3:0/3645081847 >> v2:172.21.15.157:3300/0 conn(0x7f59fc070080 0x7f59fc0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f5a00e00700 1 --2- 172.21.15.3:0/3645081847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc0adad0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f5a00e00700 1 -- 172.21.15.3:0/3645081847 >> 172.21.15.3:0/3645081847 conn(0x7f59fc0b0cf0 msgr2=0x7f59fc0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:13.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f5a00e00700 1 -- 172.21.15.3:0/3645081847 shutdown_connections 2024-09-13T16:54:13.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.386+0000 7f5a00e00700 1 -- 172.21.15.3:0/3645081847 wait complete. 2024-09-13T16:54:13.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f5a00e00700 1 Processor -- start 2024-09-13T16:54:13.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f5a00e00700 1 -- start start 2024-09-13T16:54:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f5a00e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc070080 0x7f59fc1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f5a00e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f5a00e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15cfa0 0x7f59fc15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f5a00e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc15d450 con 0x7f59fc15cfa0 2024-09-13T16:54:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f5a00e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1f6ed0 con 0x7f59fc070080 2024-09-13T16:54:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f5a00e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1f71b0 con 0x7f59fc15cbc0 2024-09-13T16:54:13.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f59fbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15cfa0 0x7f59fc15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f59fbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15cfa0 0x7f59fc15bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38904/0 (socket says 172.21.15.3:38904) 2024-09-13T16:54:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.390+0000 7f59fbe00700 1 -- 172.21.15.3:0/363215887 learned_addr learned my addr 172.21.15.3:0/363215887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59fb400700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc070080 0x7f59fc1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59faa00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59fbe00700 1 -- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 msgr2=0x7f59fc153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59fbe00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59fbe00700 1 -- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc070080 msgr2=0x7f59fc1f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59fbe00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc070080 0x7f59fc1f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59fbe00700 1 -- 172.21.15.3:0/363215887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0094b0 con 0x7f59fc15cfa0 2024-09-13T16:54:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59fb400700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc070080 0x7f59fc1f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59fbe00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15cfa0 0x7f59fc15bb00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f59f400efc0 tx=0x7f59f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59f3400700 1 -- 172.21.15.3:0/363215887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4009ea0 con 0x7f59fc15cfa0 2024-09-13T16:54:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59f3400700 1 -- 172.21.15.3:0/363215887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f400f040 con 0x7f59fc15cfa0 2024-09-13T16:54:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f59f3400700 1 -- 172.21.15.3:0/363215887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f400ced0 con 0x7f59fc15cfa0 2024-09-13T16:54:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc1f7490 con 0x7f59fc15cfa0 2024-09-13T16:54:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc1f7770 con 0x7f59fc15cfa0 2024-09-13T16:54:13.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.394+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc15d620 con 0x7f59fc15cfa0 2024-09-13T16:54:13.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.398+0000 7f59f3400700 1 -- 172.21.15.3:0/363215887 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59f4014480 con 0x7f59fc15cfa0 2024-09-13T16:54:13.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.398+0000 7f59f3400700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f59d8066730 0x7f59d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.398+0000 7f59f3400700 1 -- 172.21.15.3:0/363215887 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f59f40848a0 con 0x7f59fc15cfa0 2024-09-13T16:54:13.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.398+0000 7f59fb400700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f59d8066730 0x7f59d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.402+0000 7f59fb400700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f59d8066730 0x7f59d8068bf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0090b0 tx=0x7f59ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.406+0000 7f59f3400700 1 -- 172.21.15.3:0/363215887 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59fc15d620 con 0x7f59fc15cfa0 2024-09-13T16:54:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.594+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59fc1f7320 con 0x7f59d8066730 2024-09-13T16:54:13.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.602+0000 7f59f3400700 1 -- 172.21.15.3:0/363215887 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48646 (secure 0 0 0) 0x7f59fc1f7320 con 0x7f59d8066730 2024-09-13T16:54:13.609 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:13.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f59d8066730 msgr2=0x7f59d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f59d8066730 0x7f59d8068bf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0090b0 tx=0x7f59ec008030 comp rx=0 tx=0).stop 2024-09-13T16:54:13.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15cfa0 msgr2=0x7f59fc15bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15cfa0 0x7f59fc15bb00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f59f400efc0 tx=0x7f59f400c3a0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 shutdown_connections 2024-09-13T16:54:13.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f59d8066730 0x7f59d8068bf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15cfa0 0x7f59fc15bb00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15cbc0 0x7f59fc153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 --2- 172.21.15.3:0/363215887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc070080 0x7f59fc1f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 >> 172.21.15.3:0/363215887 conn(0x7f59fc0b0cf0 msgr2=0x7f59fc0afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 shutdown_connections 2024-09-13T16:54:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.606+0000 7f5a00e00700 1 -- 172.21.15.3:0/363215887 wait complete. 2024-09-13T16:54:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-13T16:54:13.277603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":248172,"kb_used_data":7416,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726548,"statfs":{"total":773094113280,"available":772839985152,"internally_reserved":0,"allocated":7593984,"data_stored":4110697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9374,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-77,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.691411"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T16:54:06.694045+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:54:06.694045+0000","last_peered":"2024-09-13T16:54:06.694045+0000","last_clean":"2024-09-13T16:54:06.694045+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:54:06.694045+0000","last_undegraded":"2024-09-13T16:54:06.694045+0000","last_fullsized":"2024-09-13T16:54:06.694045+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1388544,"data_stored":954329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.103,"15min":1.137},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4339999999999999,"5min":1.585,"15min":1.585},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.0860000000000001},"min":{"1min":0.78500000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5209999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.3049999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.238,"15min":1.266},"min":{"1min":0.89500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.264,"5min":1.284,"15min":1.29},"min":{"1min":0.90300000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.696,"5min":1.897,"15min":1.897},"last":1.2050000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.337,"15min":1.377},"min":{"1min":1.0289999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.7569999999999999,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.355},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2430000000000001,"15min":1.3069999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.468,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.129}]},{"osd":3,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2230000000000001,"15min":1.2589999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.4570000000000001,"5min":2.161,"15min":2.161},"last":1.173},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2829999999999999},"min":{"1min":0.86399999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.6100000000000001,"5min":1.704,"15min":1.704},"last":1.399}]},{"osd":4,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.165,"15min":1.153},"min":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.329,"15min":1.3360000000000001},"min":{"1min":0.88200000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7110000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.234}]},{"osd":5,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.361,"15min":1.363},"min":{"1min":0.71299999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.806,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.341,"15min":1.3440000000000001},"min":{"1min":0.85399999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.867,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.4159999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.371,"15min":1.3680000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.704,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.27},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4039999999999999,"15min":1.4219999999999999},"min":{"1min":0.98899999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.796,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.49}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.1279999999999999},"min":{"1min":0.74299999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.321},{"interface":"front","average":{"1min":1.129,"5min":1.222,"15min":1.3},"min":{"1min":0.76000000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.526,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":1.276}]},{"osd":1,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.232,"15min":1.3120000000000001},"min":{"1min":0.70099999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3740000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.206},"min":{"1min":0.88600000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5569999999999999,"5min":1.623,"15min":1.623},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.28,"15min":1.323},"min":{"1min":0.85299999999999998,"5min":0.745,"15min":0.745},"max":{"1min":1.661,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.343},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2509999999999999,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.462,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.3600000000000001,"15min":1.3999999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6259999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.395,"15min":1.45},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.6040000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.109}]},{"osd":4,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.2330000000000001,"15min":1.2689999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4450000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.264,"15min":1.306},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4990000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.83699999999999997}]},{"osd":5,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.266,"15min":1.3149999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6639999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.383,"15min":1.431},"min":{"1min":0.89100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6779999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.149}]},{"osd":7,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4770000000000001,"15min":1.5249999999999999},"min":{"1min":1.03,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.7110000000000001,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4430000000000001,"15min":1.5069999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8280000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.3799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31912,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.1910000000000001,"15min":1.1850000000000001},"min":{"1min":0.745,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5720000000000001,"5min":2.222,"15min":2.222},"last":1.044},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.099},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":1.601,"15min":1.601},"last":1.3109999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.395,"15min":1.4399999999999999},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.6080000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.577},{"interface":"front","average":{"1min":1.288,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.583,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.274}]},{"osd":2,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.248,"15min":1.2649999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.55,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.177},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.252,"15min":1.264},"min":{"1min":0.96199999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.611,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.212}]},{"osd":3,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.3480000000000001,"15min":1.4199999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5509999999999999,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3180000000000001,"15min":1.3360000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.643,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.532}]},{"osd":4,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.198,"15min":1.21},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4319999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.3180000000000001,"15min":1.3380000000000001},"min":{"1min":0.89300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6890000000000001,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.133}]},{"osd":6,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2829999999999999,"15min":1.3220000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.6759999999999999,"5min":1.887,"15min":1.887},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.284,"15min":1.2749999999999999},"min":{"1min":0.90100000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.657,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.494}]},{"osd":7,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.4690000000000001,"15min":1.5029999999999999},"min":{"1min":0.95699999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7549999999999999,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.649},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.474},"min":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.7410000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.081}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0760000000000001,"15min":1.0980000000000001},"min":{"1min":0.75900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3899999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.169,"15min":1.2390000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4590000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.3109999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2470000000000001,"15min":1.29},"min":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6699999999999999,"5min":1.919,"15min":1.919},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1659999999999999,"15min":1.196},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.361,"5min":1.99,"15min":1.99},"last":1.3540000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.27,"15min":1.3169999999999999},"min":{"1min":0.83499999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7070000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.462},{"interface":"front","average":{"1min":1.103,"5min":1.1850000000000001,"15min":1.2250000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4059999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.367,"15min":1.4359999999999999},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.444,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.4059999999999999,"15min":1.4750000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":1.4930000000000001,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.234}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.2529999999999999,"15min":1.3320000000000001},"min":{"1min":0.63500000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3320000000000001,"15min":1.385},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.754,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.4299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.423,"15min":1.5069999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.556,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.4990000000000001,"15min":1.601},"min":{"1min":0.74099999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.532,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.649}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4730000000000001,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5960000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.502},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7849999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.79}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95999999999999996,"15min":1.006},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4410000000000001,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2589999999999999,"5min":1.661,"15min":1.661},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.996,"15min":1.0429999999999999},"min":{"1min":0.54200000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.663,"5min":1.931,"15min":1.931},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.1399999999999999,"15min":1.1839999999999999},"min":{"1min":0.73899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.415,"5min":1.849,"15min":1.849},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.246,"15min":1.319},"min":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5640000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.3140000000000001,"15min":1.385},"min":{"1min":0.76200000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.607,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5109999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.288},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.431,"15min":1.498},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.528,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.367}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.5349999999999999,"15min":1.621},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.8180000000000001,"5min":2.79,"15min":2.79},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.516,"15min":1.611},"min":{"1min":0.96799999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5409999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.125}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.583,"15min":1.6759999999999999},"min":{"1min":1.0840000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.613,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.51,"15min":1.603},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.627,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.579,"15min":1.661},"min":{"1min":0.94799999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.6990000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.216},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.629,"15min":1.724},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7629999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.4690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.175,"5min":1.925,"15min":1.925},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.081,"15min":1.119},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3140000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.008}]},{"osd":1,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.92500000000000004,"15min":0.96299999999999997},"min":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.0369999999999999,"15min":1.073},"min":{"1min":0.57699999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.349,"5min":2.028,"15min":2.028},"last":1.196}]},{"osd":3,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.72499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6359999999999999,"5min":1.948,"15min":1.948},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.141,"15min":1.139},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.0580000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.3169999999999999},"min":{"1min":0.81599999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.577,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.123},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.3999999999999999,"15min":1.4530000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5580000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.145}]},{"osd":5,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.5,"15min":1.5680000000000001},"min":{"1min":0.93799999999999994,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.256},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3799999999999999,"15min":1.4139999999999999},"min":{"1min":0.83699999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.647,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.5069999999999999,"15min":1.5609999999999999},"min":{"1min":1.044,"5min":0.753,"15min":0.753},"max":{"1min":1.675,"5min":2.625,"15min":2.625},"last":1.494},{"interface":"front","average":{"1min":1.327,"5min":1.522,"15min":1.5900000000000001},"min":{"1min":0.91400000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.742,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.3540000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5800000000000001,"15min":1.6579999999999999},"min":{"1min":0.96899999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.681,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.597,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.872,"15min":0.872},"max":{"1min":1.722,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.528}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.61699999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2170000000000001,"5min":1.804,"15min":1.804},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.034,"5min":0.998,"15min":1},"min":{"1min":0.70699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.022}]},{"osd":2,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4419999999999999,"5min":1.954,"15min":1.954},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.042,"15min":1.0700000000000001},"min":{"1min":0.58599999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5700000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2,"15min":1.177},"min":{"1min":0.70399999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.804,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.879},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1439999999999999,"15min":1.153},"min":{"1min":0.74099999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.869,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.284,"15min":1.3140000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.752,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.28},{"interface":"front","average":{"1min":1.365,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":1.1539999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.8300000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.5,"15min":1.528},"min":{"1min":0.99299999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.0779999999999998,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.107},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4890000000000001,"15min":1.5269999999999999},"min":{"1min":1.1899999999999999,"5min":1,"15min":1},"max":{"1min":1.843,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.3460000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.504,"15min":1.536},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.3239999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.256},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.452,"15min":1.494},"min":{"1min":1.141,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.7310000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.399}]},{"osd":7,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.52,"15min":1.55},"min":{"1min":1.1899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":1.758,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.232},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.6020000000000001},"min":{"1min":1.1180000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.4390000000000001,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.155}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1388544,"data_stored":954329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.90200000000000002,"15min":0.94599999999999995},"min":{"1min":0.55600000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.2809999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.73799999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5029999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":2.6360000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3340000000000001,"5min":1.482,"15min":1.482},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.129,"15min":1.153},"min":{"1min":0.69399999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4219999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":2.7589999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.01,"15min":1.0269999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2749999999999999,"5min":1.653,"15min":1.653},"last":2.7090000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.2070000000000001,"15min":1.2330000000000001},"min":{"1min":0.73599999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.631,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.075}]},{"osd":4,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.327,"15min":1.385},"min":{"1min":0.86899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.6639999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":3.0099999999999998},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3460000000000001,"15min":1.361},"min":{"1min":0.94899999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.556,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.6699999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.446,"15min":1.498},"min":{"1min":1.1160000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8080000000000001,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":3.1110000000000002},{"interface":"front","average":{"1min":1.306,"5min":1.4139999999999999,"15min":1.468},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.7689999999999999,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":2.9279999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4910000000000001,"15min":1.5429999999999999},"min":{"1min":0.97099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7370000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.5289999999999999,"15min":1.583},"min":{"1min":1.1719999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.806,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":2.9870000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.5620000000000001,"15min":1.6220000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.9019999999999999,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":2.8889999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.4890000000000001,"15min":1.536},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.7709999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":2.8300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:54:13.634 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:54:13.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfab000700 1 Processor -- start 2024-09-13T16:54:13.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfab000700 1 -- start start 2024-09-13T16:54:13.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfab000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406ff30 0x7fcfa40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfab000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfa40b4390 0x7fcfa415a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfab000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa40cf070 con 0x7fcfa406fb50 2024-09-13T16:54:13.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfab000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa40b4be0 con 0x7fcfa40b4390 2024-09-13T16:54:13.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfab000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40cf330 con 0x7fcfa406ff30 2024-09-13T16:54:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfa9c00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcfa406fb50 0x7fcfa40cfe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56568/0 (socket says 172.21.15.3:56568) 2024-09-13T16:54:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfa9c00700 1 -- 172.21.15.3:0/2169405232 learned_addr learned my addr 172.21.15.3:0/2169405232 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.810+0000 7fcfaa600700 1 --2- 172.21.15.3:0/2169405232 >> v2:172.21.15.3:3300/0 conn(0x7fcfa40b4390 0x7fcfa415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfa9200700 1 --2- 172.21.15.3:0/2169405232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406ff30 0x7fcfa40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfa3e00700 1 -- 172.21.15.3:0/2169405232 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210966223 0 0) 0x7fcfa40cf070 con 0x7fcfa406fb50 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfa3e00700 1 -- 172.21.15.3:0/2169405232 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf8c0035d0 con 0x7fcfa406fb50 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfaa600700 1 -- 172.21.15.3:0/2169405232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406ff30 msgr2=0x7fcfa40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfaa600700 1 --2- 172.21.15.3:0/2169405232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406ff30 0x7fcfa40b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfaa600700 1 -- 172.21.15.3:0/2169405232 >> v1:172.21.15.3:6789/0 conn(0x7fcfa406fb50 legacy=0x7fcfa40cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfaa600700 1 -- 172.21.15.3:0/2169405232 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40cf610 con 0x7fcfa40b4390 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfaa600700 1 --2- 172.21.15.3:0/2169405232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa40b4390 0x7fcfa415a500 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009d90 tx=0x7fcf9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00cfb4d285c60be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfa3e00700 1 -- 172.21.15.3:0/2169405232 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00e070 con 0x7fcfa40b4390 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfa3e00700 1 -- 172.21.15.3:0/2169405232 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c00bb20 con 0x7fcfa40b4390 2024-09-13T16:54:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfa3e00700 1 -- 172.21.15.3:0/2169405232 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c012430 con 0x7fcfa40b4390 2024-09-13T16:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfab000700 1 -- 172.21.15.3:0/2169405232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa40b4390 msgr2=0x7fcfa415a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfab000700 1 --2- 172.21.15.3:0/2169405232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa40b4390 0x7fcfa415a500 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009d90 tx=0x7fcf9c00b330 comp rx=0 tx=0).stop 2024-09-13T16:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfab000700 1 -- 172.21.15.3:0/2169405232 shutdown_connections 2024-09-13T16:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfab000700 1 --2- 172.21.15.3:0/2169405232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa40b4390 0x7fcfa415a500 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfab000700 1 --2- 172.21.15.3:0/2169405232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406ff30 0x7fcfa40b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfab000700 1 -- 172.21.15.3:0/2169405232 >> 172.21.15.3:0/2169405232 conn(0x7fcfa40b2910 msgr2=0x7fcfa40b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfab000700 1 -- 172.21.15.3:0/2169405232 shutdown_connections 2024-09-13T16:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.814+0000 7fcfab000700 1 -- 172.21.15.3:0/2169405232 wait complete. 2024-09-13T16:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfab000700 1 Processor -- start 2024-09-13T16:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfab000700 1 -- start start 2024-09-13T16:54:13.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfab000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406fb50 0x7fcfa4145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfab000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa406ff30 0x7fcfa406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfab000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa40b4390 0x7fcfa41567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfab000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa41540b0 con 0x7fcfa406ff30 2024-09-13T16:54:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfab000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4153810 con 0x7fcfa40b4390 2024-09-13T16:54:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfab000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4153af0 con 0x7fcfa406fb50 2024-09-13T16:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfa9200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa406ff30 0x7fcfa406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfa9200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa406ff30 0x7fcfa406eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49520/0 (socket says 172.21.15.3:49520) 2024-09-13T16:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfa9200700 1 -- 172.21.15.3:0/619844283 learned_addr learned my addr 172.21.15.3:0/619844283 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfa9c00700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406fb50 0x7fcfa4145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.818+0000 7fcfaa600700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa40b4390 0x7fcfa41567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa9200700 1 -- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406fb50 msgr2=0x7fcfa4145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa9200700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406fb50 0x7fcfa4145960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa9200700 1 -- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa40b4390 msgr2=0x7fcfa41567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa9200700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa40b4390 0x7fcfa41567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa9200700 1 -- 172.21.15.3:0/619844283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c009a70 con 0x7fcfa406ff30 2024-09-13T16:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa9c00700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406fb50 0x7fcfa4145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa9200700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa406ff30 0x7fcfa406eda0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000e9d0 tx=0x7fcf9000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa1600700 1 -- 172.21.15.3:0/619844283 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90009720 con 0x7fcfa406ff30 2024-09-13T16:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa1600700 1 -- 172.21.15.3:0/619844283 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9000c850 con 0x7fcfa406ff30 2024-09-13T16:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfa1600700 1 -- 172.21.15.3:0/619844283 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9000cbc0 con 0x7fcfa406ff30 2024-09-13T16:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa4153dd0 con 0x7fcfa406ff30 2024-09-13T16:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa4153400 con 0x7fcfa406ff30 2024-09-13T16:54:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.822+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa4146600 con 0x7fcfa406ff30 2024-09-13T16:54:13.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.826+0000 7fcfa1600700 1 -- 172.21.15.3:0/619844283 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf9000cd20 con 0x7fcfa406ff30 2024-09-13T16:54:13.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.826+0000 7fcfa1600700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf84066760 0x7fcf84068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.826+0000 7fcfa1600700 1 -- 172.21.15.3:0/619844283 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fcf90084570 con 0x7fcfa406ff30 2024-09-13T16:54:13.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.826+0000 7fcfa9c00700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf84066760 0x7fcf84068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.830+0000 7fcfa9c00700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf84066760 0x7fcf84068c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf980011e0 tx=0x7fcf98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:13.834+0000 7fcfa1600700 1 -- 172.21.15.3:0/619844283 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcfa4146600 con 0x7fcfa406ff30 2024-09-13T16:54:14.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.022+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfa415aa40 con 0x7fcf84066760 2024-09-13T16:54:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.026+0000 7fcfa1600700 1 -- 172.21.15.3:0/619844283 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48646 (secure 0 0 0) 0x7fcf980014f0 con 0x7fcf84066760 2024-09-13T16:54:14.032 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.030+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf84066760 msgr2=0x7fcf84068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.030+0000 7fcfab000700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf84066760 0x7fcf84068c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf980011e0 tx=0x7fcf98009040 comp rx=0 tx=0).stop 2024-09-13T16:54:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.030+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa406ff30 msgr2=0x7fcfa406eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.030+0000 7fcfab000700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa406ff30 0x7fcfa406eda0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000e9d0 tx=0x7fcf9000ed90 comp rx=0 tx=0).stop 2024-09-13T16:54:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.034+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 shutdown_connections 2024-09-13T16:54:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.034+0000 7fcfab000700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fcf84066760 0x7fcf84068c20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.034+0000 7fcfab000700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa40b4390 0x7fcfa41567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.034+0000 7fcfab000700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfa406ff30 0x7fcfa406eda0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.034+0000 7fcfab000700 1 --2- 172.21.15.3:0/619844283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa406fb50 0x7fcfa4145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.034+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 >> 172.21.15.3:0/619844283 conn(0x7fcfa40b2910 msgr2=0x7fcfa406d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.034+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 shutdown_connections 2024-09-13T16:54:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.034+0000 7fcfab000700 1 -- 172.21.15.3:0/619844283 wait complete. 2024-09-13T16:54:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:54:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-13T16:54:13.277603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":248172,"kb_used_data":7416,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726548,"statfs":{"total":773094113280,"available":772839985152,"internally_reserved":0,"allocated":7593984,"data_stored":4110697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9374,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-77,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.691411"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T16:54:06.694045+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:54:06.694045+0000","last_peered":"2024-09-13T16:54:06.694045+0000","last_clean":"2024-09-13T16:54:06.694045+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:54:06.694045+0000","last_undegraded":"2024-09-13T16:54:06.694045+0000","last_fullsized":"2024-09-13T16:54:06.694045+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1388544,"data_stored":954329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.103,"15min":1.137},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4339999999999999,"5min":1.585,"15min":1.585},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.0860000000000001},"min":{"1min":0.78500000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5209999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.3049999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.238,"15min":1.266},"min":{"1min":0.89500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.264,"5min":1.284,"15min":1.29},"min":{"1min":0.90300000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.696,"5min":1.897,"15min":1.897},"last":1.2050000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.337,"15min":1.377},"min":{"1min":1.0289999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.7569999999999999,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.355},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2430000000000001,"15min":1.3069999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.468,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.129}]},{"osd":3,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2230000000000001,"15min":1.2589999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.4570000000000001,"5min":2.161,"15min":2.161},"last":1.173},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2829999999999999},"min":{"1min":0.86399999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.6100000000000001,"5min":1.704,"15min":1.704},"last":1.399}]},{"osd":4,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.165,"15min":1.153},"min":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.329,"15min":1.3360000000000001},"min":{"1min":0.88200000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7110000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.234}]},{"osd":5,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.361,"15min":1.363},"min":{"1min":0.71299999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.806,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.341,"15min":1.3440000000000001},"min":{"1min":0.85399999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.867,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.4159999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.371,"15min":1.3680000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.704,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.27},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4039999999999999,"15min":1.4219999999999999},"min":{"1min":0.98899999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.796,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.49}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.1279999999999999},"min":{"1min":0.74299999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.321},{"interface":"front","average":{"1min":1.129,"5min":1.222,"15min":1.3},"min":{"1min":0.76000000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.526,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":1.276}]},{"osd":1,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.232,"15min":1.3120000000000001},"min":{"1min":0.70099999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3740000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.206},"min":{"1min":0.88600000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5569999999999999,"5min":1.623,"15min":1.623},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.28,"15min":1.323},"min":{"1min":0.85299999999999998,"5min":0.745,"15min":0.745},"max":{"1min":1.661,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.343},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2509999999999999,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.462,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.3600000000000001,"15min":1.3999999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6259999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.395,"15min":1.45},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.6040000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.109}]},{"osd":4,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.2330000000000001,"15min":1.2689999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4450000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.264,"15min":1.306},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4990000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.83699999999999997}]},{"osd":5,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.266,"15min":1.3149999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6639999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.383,"15min":1.431},"min":{"1min":0.89100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6779999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.149}]},{"osd":7,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4770000000000001,"15min":1.5249999999999999},"min":{"1min":1.03,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.7110000000000001,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4430000000000001,"15min":1.5069999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8280000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.3799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31912,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.1910000000000001,"15min":1.1850000000000001},"min":{"1min":0.745,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5720000000000001,"5min":2.222,"15min":2.222},"last":1.044},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.099},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":1.601,"15min":1.601},"last":1.3109999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.395,"15min":1.4399999999999999},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.6080000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.577},{"interface":"front","average":{"1min":1.288,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.583,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.274}]},{"osd":2,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.248,"15min":1.2649999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.55,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.177},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.252,"15min":1.264},"min":{"1min":0.96199999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.611,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.212}]},{"osd":3,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.3480000000000001,"15min":1.4199999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5509999999999999,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3180000000000001,"15min":1.3360000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.643,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.532}]},{"osd":4,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.198,"15min":1.21},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4319999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.3180000000000001,"15min":1.3380000000000001},"min":{"1min":0.89300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6890000000000001,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.133}]},{"osd":6,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2829999999999999,"15min":1.3220000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.6759999999999999,"5min":1.887,"15min":1.887},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.284,"15min":1.2749999999999999},"min":{"1min":0.90100000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.657,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.494}]},{"osd":7,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.4690000000000001,"15min":1.5029999999999999},"min":{"1min":0.95699999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7549999999999999,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.649},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.474},"min":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.7410000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.081}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0760000000000001,"15min":1.0980000000000001},"min":{"1min":0.75900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3899999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.169,"15min":1.2390000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4590000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.3109999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2470000000000001,"15min":1.29},"min":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6699999999999999,"5min":1.919,"15min":1.919},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1659999999999999,"15min":1.196},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.361,"5min":1.99,"15min":1.99},"last":1.3540000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.27,"15min":1.3169999999999999},"min":{"1min":0.83499999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7070000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.462},{"interface":"front","average":{"1min":1.103,"5min":1.1850000000000001,"15min":1.2250000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4059999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.367,"15min":1.4359999999999999},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.444,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.4059999999999999,"15min":1.4750000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":1.4930000000000001,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.234}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.2529999999999999,"15min":1.3320000000000001},"min":{"1min":0.63500000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3320000000000001,"15min":1.385},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.754,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.4299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.423,"15min":1.5069999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.556,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.4990000000000001,"15min":1.601},"min":{"1min":0.74099999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.532,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.649}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4730000000000001,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5960000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.502},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7849999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.79}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95999999999999996,"15min":1.006},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4410000000000001,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2589999999999999,"5min":1.661,"15min":1.661},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.996,"15min":1.0429999999999999},"min":{"1min":0.54200000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.663,"5min":1.931,"15min":1.931},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.1399999999999999,"15min":1.1839999999999999},"min":{"1min":0.73899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.415,"5min":1.849,"15min":1.849},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.246,"15min":1.319},"min":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5640000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.3140000000000001,"15min":1.385},"min":{"1min":0.76200000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.607,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5109999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.288},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.431,"15min":1.498},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.528,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.367}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.5349999999999999,"15min":1.621},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.8180000000000001,"5min":2.79,"15min":2.79},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.516,"15min":1.611},"min":{"1min":0.96799999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5409999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.125}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.583,"15min":1.6759999999999999},"min":{"1min":1.0840000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.613,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.51,"15min":1.603},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.627,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.579,"15min":1.661},"min":{"1min":0.94799999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.6990000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.216},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.629,"15min":1.724},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7629999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.4690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.175,"5min":1.925,"15min":1.925},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.081,"15min":1.119},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3140000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.008}]},{"osd":1,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.92500000000000004,"15min":0.96299999999999997},"min":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.0369999999999999,"15min":1.073},"min":{"1min":0.57699999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.349,"5min":2.028,"15min":2.028},"last":1.196}]},{"osd":3,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.72499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6359999999999999,"5min":1.948,"15min":1.948},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.141,"15min":1.139},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.0580000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.3169999999999999},"min":{"1min":0.81599999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.577,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.123},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.3999999999999999,"15min":1.4530000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5580000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.145}]},{"osd":5,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.5,"15min":1.5680000000000001},"min":{"1min":0.93799999999999994,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.256},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3799999999999999,"15min":1.4139999999999999},"min":{"1min":0.83699999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.647,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.5069999999999999,"15min":1.5609999999999999},"min":{"1min":1.044,"5min":0.753,"15min":0.753},"max":{"1min":1.675,"5min":2.625,"15min":2.625},"last":1.494},{"interface":"front","average":{"1min":1.327,"5min":1.522,"15min":1.5900000000000001},"min":{"1min":0.91400000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.742,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.3540000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5800000000000001,"15min":1.6579999999999999},"min":{"1min":0.96899999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.681,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.597,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.872,"15min":0.872},"max":{"1min":1.722,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.528}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.61699999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2170000000000001,"5min":1.804,"15min":1.804},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.034,"5min":0.998,"15min":1},"min":{"1min":0.70699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.022}]},{"osd":2,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4419999999999999,"5min":1.954,"15min":1.954},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.042,"15min":1.0700000000000001},"min":{"1min":0.58599999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5700000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2,"15min":1.177},"min":{"1min":0.70399999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.804,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.879},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1439999999999999,"15min":1.153},"min":{"1min":0.74099999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.869,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.284,"15min":1.3140000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.752,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.28},{"interface":"front","average":{"1min":1.365,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":1.1539999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.8300000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.5,"15min":1.528},"min":{"1min":0.99299999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.0779999999999998,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.107},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4890000000000001,"15min":1.5269999999999999},"min":{"1min":1.1899999999999999,"5min":1,"15min":1},"max":{"1min":1.843,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.3460000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.504,"15min":1.536},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.3239999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.256},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.452,"15min":1.494},"min":{"1min":1.141,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.7310000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.399}]},{"osd":7,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.52,"15min":1.55},"min":{"1min":1.1899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":1.758,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.232},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.6020000000000001},"min":{"1min":1.1180000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.4390000000000001,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.155}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1388544,"data_stored":954329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.90200000000000002,"15min":0.94599999999999995},"min":{"1min":0.55600000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.2809999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.73799999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5029999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":2.6360000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3340000000000001,"5min":1.482,"15min":1.482},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.129,"15min":1.153},"min":{"1min":0.69399999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4219999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":2.7589999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.01,"15min":1.0269999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2749999999999999,"5min":1.653,"15min":1.653},"last":2.7090000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.2070000000000001,"15min":1.2330000000000001},"min":{"1min":0.73599999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.631,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.075}]},{"osd":4,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.327,"15min":1.385},"min":{"1min":0.86899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.6639999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":3.0099999999999998},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3460000000000001,"15min":1.361},"min":{"1min":0.94899999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.556,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.6699999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.446,"15min":1.498},"min":{"1min":1.1160000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8080000000000001,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":3.1110000000000002},{"interface":"front","average":{"1min":1.306,"5min":1.4139999999999999,"15min":1.468},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.7689999999999999,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":2.9279999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4910000000000001,"15min":1.5429999999999999},"min":{"1min":0.97099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7370000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.5289999999999999,"15min":1.583},"min":{"1min":1.1719999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.806,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":2.9870000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.5620000000000001,"15min":1.6220000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.9019999999999999,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":2.8889999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.4890000000000001,"15min":1.536},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.7709999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":2.8300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:54:14.061 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T16:54:14.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:54:14.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.234+0000 7f758ca00700 1 Processor -- start 2024-09-13T16:54:14.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.234+0000 7f758ca00700 1 -- start start 2024-09-13T16:54:14.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f758ca00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7588146ee0 0x7f75881472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f758ca00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f758806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f758ca00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7588165800 con 0x7f758806fea0 2024-09-13T16:54:14.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f758ca00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7588070a80 con 0x7f7588146ee0 2024-09-13T16:54:14.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f758ca00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75881626e0 con 0x7f758806c5e0 2024-09-13T16:54:14.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f757ea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f758806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f757ea00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f758806ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:44942/0 (socket says 172.21.15.3:44942) 2024-09-13T16:54:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f757ea00700 1 -- 172.21.15.3:0/3034711917 learned_addr learned my addr 172.21.15.3:0/3034711917 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f7587400700 1 --2- 172.21.15.3:0/3034711917 >> v2:172.21.15.157:3300/0 conn(0x7f7588146ee0 0x7f75881472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f757ea00700 1 -- 172.21.15.3:0/3034711917 >> v2:172.21.15.157:3300/0 conn(0x7f7588146ee0 msgr2=0x7f75881472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f757ea00700 1 --2- 172.21.15.3:0/3034711917 >> v2:172.21.15.157:3300/0 conn(0x7f7588146ee0 0x7f75881472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f757ea00700 1 -- 172.21.15.3:0/3034711917 >> v1:172.21.15.157:6789/0 conn(0x7f758806fea0 legacy=0x7f758806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f757ea00700 1 -- 172.21.15.3:0/3034711917 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758806cf60 con 0x7f758806c5e0 2024-09-13T16:54:14.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.238+0000 7f757ea00700 1 --2- 172.21.15.3:0/3034711917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f758806ca20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7570009910 tx=0x7f7570009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4bc72bf8d33e006 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:14.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f7586a00700 1 -- 172.21.15.3:0/3034711917 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000ba60 con 0x7f758806c5e0 2024-09-13T16:54:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f7586a00700 1 -- 172.21.15.3:0/3034711917 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757000bbc0 con 0x7f758806c5e0 2024-09-13T16:54:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f7586a00700 1 -- 172.21.15.3:0/3034711917 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570009e60 con 0x7f758806c5e0 2024-09-13T16:54:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f758ca00700 1 -- 172.21.15.3:0/3034711917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 msgr2=0x7f758806ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f758ca00700 1 --2- 172.21.15.3:0/3034711917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f758806ca20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7570009910 tx=0x7f7570009c20 comp rx=0 tx=0).stop 2024-09-13T16:54:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f758ca00700 1 -- 172.21.15.3:0/3034711917 shutdown_connections 2024-09-13T16:54:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f758ca00700 1 --2- 172.21.15.3:0/3034711917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f758806ca20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f758ca00700 1 --2- 172.21.15.3:0/3034711917 >> v2:172.21.15.157:3300/0 conn(0x7f7588146ee0 0x7f75881472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f758ca00700 1 -- 172.21.15.3:0/3034711917 >> 172.21.15.3:0/3034711917 conn(0x7f75880b0cf0 msgr2=0x7f75880b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f758ca00700 1 -- 172.21.15.3:0/3034711917 shutdown_connections 2024-09-13T16:54:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.242+0000 7f758ca00700 1 -- 172.21.15.3:0/3034711917 wait complete. 2024-09-13T16:54:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f758ca00700 1 Processor -- start 2024-09-13T16:54:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f758ca00700 1 -- start start 2024-09-13T16:54:14.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f758ca00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f75881eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f758ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758806fea0 0x7f75881465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f758ca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7588146ee0 0x7f758815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f758ca00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7588146ae0 con 0x7f7588146ee0 2024-09-13T16:54:14.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f758ca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758815b8b0 con 0x7f758806fea0 2024-09-13T16:54:14.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f758ca00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7588140ce0 con 0x7f758806c5e0 2024-09-13T16:54:14.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f7587e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7588146ee0 0x7f758815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f7587e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7588146ee0 0x7f758815b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49536/0 (socket says 172.21.15.3:49536) 2024-09-13T16:54:14.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f7587e00700 1 -- 172.21.15.3:0/1030388652 learned_addr learned my addr 172.21.15.3:0/1030388652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f757ea00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758806fea0 0x7f75881465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.246+0000 7f7587400700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f75881eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f7587e00700 1 -- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 msgr2=0x7f75881eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f7587e00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f75881eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f7587e00700 1 -- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758806fea0 msgr2=0x7f75881465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f7587e00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758806fea0 0x7f75881465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f7587e00700 1 -- 172.21.15.3:0/1030388652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570009570 con 0x7f7588146ee0 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f757ea00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758806fea0 0x7f75881465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:14.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f7587e00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7588146ee0 0x7f758815b370 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f758000c4b0 tx=0x7f758000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:14.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f757fe00700 1 -- 172.21.15.3:0/1030388652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800052d0 con 0x7f7588146ee0 2024-09-13T16:54:14.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f757fe00700 1 -- 172.21.15.3:0/1030388652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7580005430 con 0x7f7588146ee0 2024-09-13T16:54:14.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f757fe00700 1 -- 172.21.15.3:0/1030388652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800055a0 con 0x7f7588146ee0 2024-09-13T16:54:14.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75881ebd80 con 0x7f7588146ee0 2024-09-13T16:54:14.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7588141230 con 0x7f7588146ee0 2024-09-13T16:54:14.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.250+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758806ca20 con 0x7f7588146ee0 2024-09-13T16:54:14.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.254+0000 7f757fe00700 1 -- 172.21.15.3:0/1030388652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75800038e0 con 0x7f7588146ee0 2024-09-13T16:54:14.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.254+0000 7f757fe00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7564066760 0x7f7564068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.254+0000 7f757fe00700 1 -- 172.21.15.3:0/1030388652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f7580083460 con 0x7f7588146ee0 2024-09-13T16:54:14.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.254+0000 7f7587400700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7564066760 0x7f7564068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.258+0000 7f7587400700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7564066760 0x7f7564068c20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7578003b50 tx=0x7f7578009830 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:14.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.262+0000 7f757fe00700 1 -- 172.21.15.3:0/1030388652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7580050840 con 0x7f7588146ee0 2024-09-13T16:54:14.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.454+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7588165490 con 0x7f7564066760 2024-09-13T16:54:14.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.462+0000 7f757fe00700 1 -- 172.21.15.3:0/1030388652 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48646 (secure 0 0 0) 0x7f7578004510 con 0x7f7564066760 2024-09-13T16:54:14.466 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7564066760 msgr2=0x7f7564068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7564066760 0x7f7564068c20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7578003b50 tx=0x7f7578009830 comp rx=0 tx=0).stop 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7588146ee0 msgr2=0x7f758815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7588146ee0 0x7f758815b370 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f758000c4b0 tx=0x7f758000c7c0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 shutdown_connections 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7564066760 0x7f7564068c20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7588146ee0 0x7f758815b370 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758806fea0 0x7f75881465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 --2- 172.21.15.3:0/1030388652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758806c5e0 0x7f75881eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 >> 172.21.15.3:0/1030388652 conn(0x7f75880b0cf0 msgr2=0x7f75880cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 shutdown_connections 2024-09-13T16:54:14.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.466+0000 7f758ca00700 1 -- 172.21.15.3:0/1030388652 wait complete. 2024-09-13T16:54:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:54:14.497 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-13T16:54:13.277603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":248172,"kb_used_data":7416,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726548,"statfs":{"total":773094113280,"available":772839985152,"internally_reserved":0,"allocated":7593984,"data_stored":4110697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9374,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-77,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.691411"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T16:54:06.694045+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:54:06.694045+0000","last_peered":"2024-09-13T16:54:06.694045+0000","last_clean":"2024-09-13T16:54:06.694045+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:54:06.694045+0000","last_undegraded":"2024-09-13T16:54:06.694045+0000","last_fullsized":"2024-09-13T16:54:06.694045+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1388544,"data_stored":954329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.103,"15min":1.137},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4339999999999999,"5min":1.585,"15min":1.585},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.0860000000000001},"min":{"1min":0.78500000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5209999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.3049999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.238,"15min":1.266},"min":{"1min":0.89500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.264,"5min":1.284,"15min":1.29},"min":{"1min":0.90300000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.696,"5min":1.897,"15min":1.897},"last":1.2050000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.337,"15min":1.377},"min":{"1min":1.0289999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.7569999999999999,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.355},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2430000000000001,"15min":1.3069999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.468,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.129}]},{"osd":3,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2230000000000001,"15min":1.2589999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.4570000000000001,"5min":2.161,"15min":2.161},"last":1.173},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2829999999999999},"min":{"1min":0.86399999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.6100000000000001,"5min":1.704,"15min":1.704},"last":1.399}]},{"osd":4,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.165,"15min":1.153},"min":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.329,"15min":1.3360000000000001},"min":{"1min":0.88200000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7110000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.234}]},{"osd":5,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.361,"15min":1.363},"min":{"1min":0.71299999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.806,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.341,"15min":1.3440000000000001},"min":{"1min":0.85399999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.867,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.4159999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.371,"15min":1.3680000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.704,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.27},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4039999999999999,"15min":1.4219999999999999},"min":{"1min":0.98899999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.796,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.49}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.1279999999999999},"min":{"1min":0.74299999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.321},{"interface":"front","average":{"1min":1.129,"5min":1.222,"15min":1.3},"min":{"1min":0.76000000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.526,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":1.276}]},{"osd":1,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.232,"15min":1.3120000000000001},"min":{"1min":0.70099999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3740000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.206},"min":{"1min":0.88600000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5569999999999999,"5min":1.623,"15min":1.623},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.28,"15min":1.323},"min":{"1min":0.85299999999999998,"5min":0.745,"15min":0.745},"max":{"1min":1.661,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.343},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2509999999999999,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.462,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.3600000000000001,"15min":1.3999999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6259999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.395,"15min":1.45},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.6040000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.109}]},{"osd":4,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.2330000000000001,"15min":1.2689999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4450000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.264,"15min":1.306},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4990000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.83699999999999997}]},{"osd":5,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.266,"15min":1.3149999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6639999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.383,"15min":1.431},"min":{"1min":0.89100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6779999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.149}]},{"osd":7,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4770000000000001,"15min":1.5249999999999999},"min":{"1min":1.03,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.7110000000000001,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4430000000000001,"15min":1.5069999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8280000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.3799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31912,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.1910000000000001,"15min":1.1850000000000001},"min":{"1min":0.745,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5720000000000001,"5min":2.222,"15min":2.222},"last":1.044},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.099},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":1.601,"15min":1.601},"last":1.3109999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.395,"15min":1.4399999999999999},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.6080000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.577},{"interface":"front","average":{"1min":1.288,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.583,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.274}]},{"osd":2,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.248,"15min":1.2649999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.55,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.177},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.252,"15min":1.264},"min":{"1min":0.96199999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.611,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.212}]},{"osd":3,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.3480000000000001,"15min":1.4199999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5509999999999999,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3180000000000001,"15min":1.3360000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.643,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.532}]},{"osd":4,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.198,"15min":1.21},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4319999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.3180000000000001,"15min":1.3380000000000001},"min":{"1min":0.89300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6890000000000001,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.133}]},{"osd":6,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2829999999999999,"15min":1.3220000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.6759999999999999,"5min":1.887,"15min":1.887},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.284,"15min":1.2749999999999999},"min":{"1min":0.90100000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.657,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.494}]},{"osd":7,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.4690000000000001,"15min":1.5029999999999999},"min":{"1min":0.95699999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7549999999999999,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.649},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.474},"min":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.7410000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.081}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0760000000000001,"15min":1.0980000000000001},"min":{"1min":0.75900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3899999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.169,"15min":1.2390000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4590000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.3109999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2470000000000001,"15min":1.29},"min":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6699999999999999,"5min":1.919,"15min":1.919},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1659999999999999,"15min":1.196},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.361,"5min":1.99,"15min":1.99},"last":1.3540000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.27,"15min":1.3169999999999999},"min":{"1min":0.83499999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7070000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.462},{"interface":"front","average":{"1min":1.103,"5min":1.1850000000000001,"15min":1.2250000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4059999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.367,"15min":1.4359999999999999},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.444,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.4059999999999999,"15min":1.4750000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":1.4930000000000001,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.234}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.2529999999999999,"15min":1.3320000000000001},"min":{"1min":0.63500000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3320000000000001,"15min":1.385},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.754,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.4299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.423,"15min":1.5069999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.556,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.4990000000000001,"15min":1.601},"min":{"1min":0.74099999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.532,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.649}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4730000000000001,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5960000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.502},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7849999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.79}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95999999999999996,"15min":1.006},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4410000000000001,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2589999999999999,"5min":1.661,"15min":1.661},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.996,"15min":1.0429999999999999},"min":{"1min":0.54200000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.663,"5min":1.931,"15min":1.931},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.1399999999999999,"15min":1.1839999999999999},"min":{"1min":0.73899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.415,"5min":1.849,"15min":1.849},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.246,"15min":1.319},"min":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5640000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.3140000000000001,"15min":1.385},"min":{"1min":0.76200000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.607,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5109999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.288},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.431,"15min":1.498},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.528,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.367}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.5349999999999999,"15min":1.621},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.8180000000000001,"5min":2.79,"15min":2.79},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.516,"15min":1.611},"min":{"1min":0.96799999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5409999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.125}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.583,"15min":1.6759999999999999},"min":{"1min":1.0840000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.613,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.51,"15min":1.603},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.627,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.579,"15min":1.661},"min":{"1min":0.94799999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.6990000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.216},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.629,"15min":1.724},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7629999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.4690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":794624,"data_stored":363961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.175,"5min":1.925,"15min":1.925},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.081,"15min":1.119},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3140000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.008}]},{"osd":1,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.92500000000000004,"15min":0.96299999999999997},"min":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.0369999999999999,"15min":1.073},"min":{"1min":0.57699999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.349,"5min":2.028,"15min":2.028},"last":1.196}]},{"osd":3,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.72499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6359999999999999,"5min":1.948,"15min":1.948},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.141,"15min":1.139},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.0580000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.3169999999999999},"min":{"1min":0.81599999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.577,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.123},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.3999999999999999,"15min":1.4530000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5580000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.145}]},{"osd":5,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.5,"15min":1.5680000000000001},"min":{"1min":0.93799999999999994,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.256},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3799999999999999,"15min":1.4139999999999999},"min":{"1min":0.83699999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.647,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.5069999999999999,"15min":1.5609999999999999},"min":{"1min":1.044,"5min":0.753,"15min":0.753},"max":{"1min":1.675,"5min":2.625,"15min":2.625},"last":1.494},{"interface":"front","average":{"1min":1.327,"5min":1.522,"15min":1.5900000000000001},"min":{"1min":0.91400000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.742,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.3540000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5800000000000001,"15min":1.6579999999999999},"min":{"1min":0.96899999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.681,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.597,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.872,"15min":0.872},"max":{"1min":1.722,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.528}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.61699999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2170000000000001,"5min":1.804,"15min":1.804},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.034,"5min":0.998,"15min":1},"min":{"1min":0.70699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.022}]},{"osd":2,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4419999999999999,"5min":1.954,"15min":1.954},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.042,"15min":1.0700000000000001},"min":{"1min":0.58599999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5700000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2,"15min":1.177},"min":{"1min":0.70399999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.804,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.879},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1439999999999999,"15min":1.153},"min":{"1min":0.74099999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.869,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.284,"15min":1.3140000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.752,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.28},{"interface":"front","average":{"1min":1.365,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":1.1539999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.8300000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.5,"15min":1.528},"min":{"1min":0.99299999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.0779999999999998,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.107},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4890000000000001,"15min":1.5269999999999999},"min":{"1min":1.1899999999999999,"5min":1,"15min":1},"max":{"1min":1.843,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.3460000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.504,"15min":1.536},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.3239999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.256},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.452,"15min":1.494},"min":{"1min":1.141,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.7310000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.399}]},{"osd":7,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.52,"15min":1.55},"min":{"1min":1.1899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":1.758,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.232},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.6020000000000001},"min":{"1min":1.1180000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.4390000000000001,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.155}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1388544,"data_stored":954329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.90200000000000002,"15min":0.94599999999999995},"min":{"1min":0.55600000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.2809999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.73799999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5029999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":2.6360000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3340000000000001,"5min":1.482,"15min":1.482},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.129,"15min":1.153},"min":{"1min":0.69399999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4219999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":2.7589999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.01,"15min":1.0269999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2749999999999999,"5min":1.653,"15min":1.653},"last":2.7090000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.2070000000000001,"15min":1.2330000000000001},"min":{"1min":0.73599999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.631,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.075}]},{"osd":4,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.327,"15min":1.385},"min":{"1min":0.86899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.6639999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":3.0099999999999998},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3460000000000001,"15min":1.361},"min":{"1min":0.94899999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.556,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.6699999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.446,"15min":1.498},"min":{"1min":1.1160000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8080000000000001,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":3.1110000000000002},{"interface":"front","average":{"1min":1.306,"5min":1.4139999999999999,"15min":1.468},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.7689999999999999,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":2.9279999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4910000000000001,"15min":1.5429999999999999},"min":{"1min":0.97099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7370000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.5289999999999999,"15min":1.583},"min":{"1min":1.1719999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.806,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":2.9870000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.5620000000000001,"15min":1.6220000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.9019999999999999,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":2.8889999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.4890000000000001,"15min":1.536},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.7709999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":2.8300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:54:14.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:54:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.674+0000 7f7266e00700 1 Processor -- start 2024-09-13T16:54:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.674+0000 7f7266e00700 1 -- start start 2024-09-13T16:54:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7260071de0 0x7f72601462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7260146c80 con 0x7f72600721c0 2024-09-13T16:54:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72600b4b40 con 0x7f7260146810 2024-09-13T16:54:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72600b3aa0 con 0x7f7260071de0 2024-09-13T16:54:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7260071de0 0x7f72601462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7260071de0 0x7f72601462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49548/0 (socket says 172.21.15.3:49548) 2024-09-13T16:54:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725fe00700 1 -- 172.21.15.3:0/685350070 learned_addr learned my addr 172.21.15.3:0/685350070 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725f400700 1 --1- 172.21.15.3:0/685350070 >> v1:172.21.15.3:6789/0 conn(0x7f72600721c0 0x7f7260160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56578/0 (socket says 172.21.15.3:56578) 2024-09-13T16:54:14.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725fe00700 1 -- 172.21.15.3:0/685350070 >> v1:172.21.15.157:6789/0 conn(0x7f7260146810 legacy=0x7f726006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:14.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725fe00700 1 -- 172.21.15.3:0/685350070 >> v1:172.21.15.3:6789/0 conn(0x7f72600721c0 legacy=0x7f7260160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725fe00700 1 -- 172.21.15.3:0/685350070 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600b3d80 con 0x7f7260071de0 2024-09-13T16:54:14.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725fe00700 1 --2- 172.21.15.3:0/685350070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7260071de0 0x7f72601462d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7250009910 tx=0x7f7250009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82ffa3fcaf0e737 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:14.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725ea00700 1 -- 172.21.15.3:0/685350070 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250004230 con 0x7f7260071de0 2024-09-13T16:54:14.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725ea00700 1 -- 172.21.15.3:0/685350070 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7250004390 con 0x7f7260071de0 2024-09-13T16:54:14.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f725ea00700 1 -- 172.21.15.3:0/685350070 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250009e60 con 0x7f7260071de0 2024-09-13T16:54:14.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 -- 172.21.15.3:0/685350070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7260071de0 msgr2=0x7f72601462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 --2- 172.21.15.3:0/685350070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7260071de0 0x7f72601462d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7250009910 tx=0x7f7250009c20 comp rx=0 tx=0).stop 2024-09-13T16:54:14.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 -- 172.21.15.3:0/685350070 shutdown_connections 2024-09-13T16:54:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 --2- 172.21.15.3:0/685350070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7260071de0 0x7f72601462d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 -- 172.21.15.3:0/685350070 >> 172.21.15.3:0/685350070 conn(0x7f7260071050 msgr2=0x7f7260071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 -- 172.21.15.3:0/685350070 shutdown_connections 2024-09-13T16:54:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.678+0000 7f7266e00700 1 -- 172.21.15.3:0/685350070 wait complete. 2024-09-13T16:54:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f7266e00700 1 Processor -- start 2024-09-13T16:54:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f7266e00700 1 -- start start 2024-09-13T16:54:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f7266e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7260071de0 0x7f72600b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f7266e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72600721c0 0x7f72600b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f7266e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7260146810 0x7f72601333c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f7266e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7260143c20 con 0x7f72600721c0 2024-09-13T16:54:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f7266e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72600ad690 con 0x7f7260146810 2024-09-13T16:54:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f7266e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7260131b20 con 0x7f7260071de0 2024-09-13T16:54:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f725f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72600721c0 0x7f72600b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f725f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72600721c0 0x7f72600b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49552/0 (socket says 172.21.15.3:49552) 2024-09-13T16:54:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.682+0000 7f725f400700 1 -- 172.21.15.3:0/364001316 learned_addr learned my addr 172.21.15.3:0/364001316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f7264e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7260146810 0x7f72601333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f725fe00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7260071de0 0x7f72600b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f725f400700 1 -- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7260071de0 msgr2=0x7f72600b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f725f400700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7260071de0 0x7f72600b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f725f400700 1 -- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7260146810 msgr2=0x7f72601333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f725f400700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7260146810 0x7f72601333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f725f400700 1 -- 172.21.15.3:0/364001316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250009570 con 0x7f72600721c0 2024-09-13T16:54:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f7264e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7260146810 0x7f72601333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f725f400700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72600721c0 0x7f72600b0590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c530 tx=0x7f724c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f7257e00700 1 -- 172.21.15.3:0/364001316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c005cd0 con 0x7f72600721c0 2024-09-13T16:54:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f7257e00700 1 -- 172.21.15.3:0/364001316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c005e30 con 0x7f72600721c0 2024-09-13T16:54:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726015cd50 con 0x7f72600721c0 2024-09-13T16:54:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f7257e00700 1 -- 172.21.15.3:0/364001316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0072f0 con 0x7f72600721c0 2024-09-13T16:54:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72601314d0 con 0x7f72600721c0 2024-09-13T16:54:14.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.686+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7260131330 con 0x7f72600721c0 2024-09-13T16:54:14.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.690+0000 7f7257e00700 1 -- 172.21.15.3:0/364001316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f724c007450 con 0x7f72600721c0 2024-09-13T16:54:14.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.690+0000 7f7257e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7240066730 0x7f7240068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:14.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.690+0000 7f7257e00700 1 -- 172.21.15.3:0/364001316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f724c083400 con 0x7f72600721c0 2024-09-13T16:54:14.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.690+0000 7f725fe00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7240066730 0x7f7240068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:14.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.694+0000 7f725fe00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7240066730 0x7f7240068bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7250009910 tx=0x7f7250008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.698+0000 7f7257e00700 1 -- 172.21.15.3:0/364001316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7260131330 con 0x7f72600721c0 2024-09-13T16:54:14.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.886+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7260146c80 con 0x7f72600721c0 2024-09-13T16:54:14.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.886+0000 7f7257e00700 1 -- 172.21.15.3:0/364001316 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+13399 (secure 0 0 0) 0x7f7260146c80 con 0x7f72600721c0 2024-09-13T16:54:14.892 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:14.892 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":67,"fsid":"61456116-a42d-4065-9bf5-a1daf0be594e","created":"2024-09-13T16:50:06.742018+0000","modified":"2024-09-13T16:54:08.606272+0000","last_up_change":"2024-09-13T16:50:18.985693+0000","last_in_change":"2024-09-13T16:50:09.694922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:50:19.278361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1db2a65-68c9-405c-bbfa-f9ee9a87343e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6825","nonce":3820836136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6827","nonce":3820836136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6831","nonce":3820836136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3820836136},{"type":"v1","addr":"172.21.15.3:6829","nonce":3820836136}]},"public_addr":"172.21.15.3:6825/3820836136","cluster_addr":"172.21.15.3:6827/3820836136","heartbeat_back_addr":"172.21.15.3:6831/3820836136","heartbeat_front_addr":"172.21.15.3:6829/3820836136","state":["exists","up"]},{"osd":1,"uuid":"02190853-86bc-48c1-b718-2b150b722955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6801","nonce":4234138253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6803","nonce":4234138253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6807","nonce":4234138253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4234138253},{"type":"v1","addr":"172.21.15.3:6805","nonce":4234138253}]},"public_addr":"172.21.15.3:6801/4234138253","cluster_addr":"172.21.15.3:6803/4234138253","heartbeat_back_addr":"172.21.15.3:6807/4234138253","heartbeat_front_addr":"172.21.15.3:6805/4234138253","state":["exists","up"]},{"osd":2,"uuid":"321a1109-12e8-4ce4-b73a-9971e6a42760","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6809","nonce":4113809917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6811","nonce":4113809917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6815","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6818","nonce":4113809917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4113809917},{"type":"v1","addr":"172.21.15.3:6814","nonce":4113809917}]},"public_addr":"172.21.15.3:6809/4113809917","cluster_addr":"172.21.15.3:6811/4113809917","heartbeat_back_addr":"172.21.15.3:6818/4113809917","heartbeat_front_addr":"172.21.15.3:6814/4113809917","state":["exists","up"]},{"osd":3,"uuid":"35421139-a6b7-48fb-a6cd-4521d2746666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6813","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6816","nonce":1788023973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6819","nonce":1788023973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6823","nonce":1788023973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1788023973},{"type":"v1","addr":"172.21.15.3:6821","nonce":1788023973}]},"public_addr":"172.21.15.3:6816/1788023973","cluster_addr":"172.21.15.3:6819/1788023973","heartbeat_back_addr":"172.21.15.3:6823/1788023973","heartbeat_front_addr":"172.21.15.3:6821/1788023973","state":["exists","up"]},{"osd":4,"uuid":"a4cb098f-d503-43f5-849a-ed8049e85047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6809","nonce":960819469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6811","nonce":960819469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6818","nonce":960819469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":960819469},{"type":"v1","addr":"172.21.15.157:6815","nonce":960819469}]},"public_addr":"172.21.15.157:6809/960819469","cluster_addr":"172.21.15.157:6811/960819469","heartbeat_back_addr":"172.21.15.157:6818/960819469","heartbeat_front_addr":"172.21.15.157:6815/960819469","state":["exists","up"]},{"osd":5,"uuid":"8df79f9b-285f-4f73-aac8-9b98c1554a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6801","nonce":2560288896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6803","nonce":2560288896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6807","nonce":2560288896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2560288896},{"type":"v1","addr":"172.21.15.157:6805","nonce":2560288896}]},"public_addr":"172.21.15.157:6801/2560288896","cluster_addr":"172.21.15.157:6803/2560288896","heartbeat_back_addr":"172.21.15.157:6807/2560288896","heartbeat_front_addr":"172.21.15.157:6805/2560288896","state":["exists","up"]},{"osd":6,"uuid":"afc3b2ec-4bf7-4ac7-878c-4da7f21a6d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6825","nonce":1906982980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6827","nonce":1906982980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6831","nonce":1906982980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1906982980},{"type":"v1","addr":"172.21.15.157:6829","nonce":1906982980}]},"public_addr":"172.21.15.157:6825/1906982980","cluster_addr":"172.21.15.157:6827/1906982980","heartbeat_back_addr":"172.21.15.157:6831/1906982980","heartbeat_front_addr":"172.21.15.157:6829/1906982980","state":["exists","up"]},{"osd":7,"uuid":"edeb379b-9a58-440e-911d-fd3119dd838d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6814","nonce":3325903215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6817","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6819","nonce":3325903215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6823","nonce":3325903215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3325903215},{"type":"v1","addr":"172.21.15.157:6821","nonce":3325903215}]},"public_addr":"172.21.15.157:6814/3325903215","cluster_addr":"172.21.15.157:6819/3325903215","heartbeat_back_addr":"172.21.15.157:6823/3325903215","heartbeat_front_addr":"172.21.15.157:6821/3325903215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.445191+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.495968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.465365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.500911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.561271+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.466821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.582039+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:50:15.487073+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/2114372521":"2024-09-14T16:54:05.581543+0000","172.21.15.3:6834/2114372521":"2024-09-14T16:54:05.581543+0000","172.21.15.157:6834/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.157:6836/2335957931":"2024-09-14T16:53:08.803287+0000","172.21.15.3:6835/291442393":"2024-09-14T16:53:08.782245+0000","172.21.15.3:6834/291442393":"2024-09-14T16:53:08.782245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:54:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7240066730 msgr2=0x7f7240068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7240066730 0x7f7240068bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7250009910 tx=0x7f7250008030 comp rx=0 tx=0).stop 2024-09-13T16:54:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72600721c0 msgr2=0x7f72600b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72600721c0 0x7f72600b0590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c530 tx=0x7f724c00c840 comp rx=0 tx=0).stop 2024-09-13T16:54:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 shutdown_connections 2024-09-13T16:54:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7240066730 0x7f7240068bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7260146810 0x7f72601333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72600721c0 0x7f72600b0590 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 --2- 172.21.15.3:0/364001316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7260071de0 0x7f72600b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:14.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 >> 172.21.15.3:0/364001316 conn(0x7f7260071050 msgr2=0x7f726015e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:14.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 shutdown_connections 2024-09-13T16:54:14.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:14.894+0000 7f7266e00700 1 -- 172.21.15.3:0/364001316 wait complete. 2024-09-13T16:54:15.920 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T16:54:15.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T16:54:16.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.106+0000 7f60d1400700 1 Processor -- start 2024-09-13T16:54:16.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.106+0000 7f60d1400700 1 -- start start 2024-09-13T16:54:16.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60d1400700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60cc146580 0x7f60cc0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60d1400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc0b4c40 0x7f60cc0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60d1400700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60cc0ce6a0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60d1400700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc0b5080 con 0x7f60cc146580 2024-09-13T16:54:16.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60d1400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0ce090 con 0x7f60cc0b4c40 2024-09-13T16:54:16.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60cb400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc0b4c40 0x7f60cc0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60d0a00700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f60cc0b17f0 0x7f60cc06c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:46062/0 (socket says 172.21.15.3:46062) 2024-09-13T16:54:16.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60d0a00700 1 -- 172.21.15.3:0/529720604 learned_addr learned my addr 172.21.15.3:0/529720604 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:16.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60cbe00700 1 --2- 172.21.15.3:0/529720604 >> v2:172.21.15.157:3300/0 conn(0x7f60cc146580 0x7f60cc0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60caa00700 1 -- 172.21.15.3:0/529720604 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479833321 0 0) 0x7f60cc0ce6a0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60caa00700 1 -- 172.21.15.3:0/529720604 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60b00035d0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60cbe00700 1 -- 172.21.15.3:0/529720604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc0b4c40 msgr2=0x7f60cc0b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:16.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60cbe00700 1 --2- 172.21.15.3:0/529720604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc0b4c40 0x7f60cc0b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60cbe00700 1 -- 172.21.15.3:0/529720604 >> v1:172.21.15.157:6789/0 conn(0x7f60cc0b17f0 legacy=0x7f60cc06c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60cbe00700 1 -- 172.21.15.3:0/529720604 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc0b2010 con 0x7f60cc146580 2024-09-13T16:54:16.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60cb400700 1 --2- 172.21.15.3:0/529720604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc0b4c40 0x7f60cc0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:16.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.110+0000 7f60cbe00700 1 --2- 172.21.15.3:0/529720604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60cc146580 0x7f60cc0b4700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009850 tx=0x7f60bc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91c8d59be95f35e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60caa00700 1 -- 172.21.15.3:0/529720604 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc011830 con 0x7f60cc146580 2024-09-13T16:54:16.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60caa00700 1 -- 172.21.15.3:0/529720604 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc011990 con 0x7f60cc146580 2024-09-13T16:54:16.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60caa00700 1 -- 172.21.15.3:0/529720604 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc011d40 con 0x7f60cc146580 2024-09-13T16:54:16.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60d1400700 1 -- 172.21.15.3:0/529720604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60cc146580 msgr2=0x7f60cc0b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60d1400700 1 --2- 172.21.15.3:0/529720604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60cc146580 0x7f60cc0b4700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009850 tx=0x7f60bc009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:16.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60d1400700 1 -- 172.21.15.3:0/529720604 shutdown_connections 2024-09-13T16:54:16.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60d1400700 1 --2- 172.21.15.3:0/529720604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc0b4c40 0x7f60cc0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60d1400700 1 --2- 172.21.15.3:0/529720604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60cc146580 0x7f60cc0b4700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60d1400700 1 -- 172.21.15.3:0/529720604 >> 172.21.15.3:0/529720604 conn(0x7f60cc0b6210 msgr2=0x7f60cc0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:16.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60d1400700 1 -- 172.21.15.3:0/529720604 shutdown_connections 2024-09-13T16:54:16.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.114+0000 7f60d1400700 1 -- 172.21.15.3:0/529720604 wait complete. 2024-09-13T16:54:16.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d1400700 1 Processor -- start 2024-09-13T16:54:16.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d1400700 1 -- start start 2024-09-13T16:54:16.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d1400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60cc0b17f0 0x7f60cc15c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d1400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60cc0b4c40 0x7f60cc15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d1400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc146580 0x7f60cc1f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d1400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc15cbd0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d1400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc15d880 con 0x7f60cc0b4c40 2024-09-13T16:54:16.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d1400700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc1f36c0 con 0x7f60cc146580 2024-09-13T16:54:16.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60cc0b17f0 0x7f60cc15c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60cc0b17f0 0x7f60cc15c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49562/0 (socket says 172.21.15.3:49562) 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 -- 172.21.15.3:0/718643280 learned_addr learned my addr 172.21.15.3:0/718643280 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cb400700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60cc0b4c40 0x7f60cc15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60d0a00700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc146580 0x7f60cc1f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 -- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc146580 msgr2=0x7f60cc1f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60cc146580 0x7f60cc1f3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 -- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60cc0b4c40 msgr2=0x7f60cc15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60cc0b4c40 0x7f60cc15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 -- 172.21.15.3:0/718643280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b80094b0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.118+0000 7f60cbe00700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60cc0b17f0 0x7f60cc15c690 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f60bc014040 tx=0x7f60bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.122+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc0036c0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.122+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc01e070 con 0x7f60cc0b17f0 2024-09-13T16:54:16.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.122+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc003a80 con 0x7f60cc0b17f0 2024-09-13T16:54:16.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.122+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc0094b0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.122+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc0b1de0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.122+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60cc0b1c60 con 0x7f60cc0b17f0 2024-09-13T16:54:16.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.122+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f60bc003be0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60c3e00700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f60ac066730 0x7f60ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f60bc020070 con 0x7f60cc0b17f0 2024-09-13T16:54:16.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60c3e00700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7f60ac06cb70 0x7f60ac06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 --> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60ac06f680 con 0x7f60ac06cb70 2024-09-13T16:54:16.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60bc0879e0 con 0x7f60cc0b17f0 2024-09-13T16:54:16.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60cb400700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f60ac066730 0x7f60ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60d0a00700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7f60ac06cb70 0x7f60ac06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60cb400700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f60ac066730 0x7f60ac068bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f60cc160100 tx=0x7f60b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.126+0000 7f60d0a00700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7f60ac06cb70 0x7f60ac06efb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.130+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 <== osd.0 v2:172.21.15.3:6824/3820836136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f60ac06f680 con 0x7f60ac06cb70 2024-09-13T16:54:16.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 --> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f60cc0ce330 con 0x7f60ac06cb70 2024-09-13T16:54:16.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60c3e00700 1 -- 172.21.15.3:0/718643280 <== osd.0 v2:172.21.15.3:6824/3820836136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f60c40070e0 con 0x7f60ac06cb70 2024-09-13T16:54:16.152 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:54:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T16:54:16.153 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7f60ac06cb70 msgr2=0x7f60ac06efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.3:6824/3820836136,v1:172.21.15.3:6825/3820836136] conn(0x7f60ac06cb70 0x7f60ac06efb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f60ac066730 msgr2=0x7f60ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f60ac066730 0x7f60ac068bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f60cc160100 tx=0x7f60b8008040 comp rx=0 tx=0).stop 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60cc0b17f0 msgr2=0x7f60cc15c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 --2- 172.21.15.3:0/718643280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60cc0b17f0 0x7f60cc15c690 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f60bc014040 tx=0x7f60bc004770 comp rx=0 tx=0).stop 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d0a00700 1 -- 172.21.15.3:0/718643280 reap_dead start 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 shutdown_connections 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 >> 172.21.15.3:0/718643280 conn(0x7f60cc0b6210 msgr2=0x7f60cc0ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 shutdown_connections 2024-09-13T16:54:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.150+0000 7f60d1400700 1 -- 172.21.15.3:0/718643280 wait complete. 2024-09-13T16:54:16.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T16:54:16.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.358+0000 7ff797200700 1 Processor -- start 2024-09-13T16:54:16.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.358+0000 7ff797200700 1 -- start start 2024-09-13T16:54:16.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff797200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff79806fb50 0x7ff7980cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff797200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806ff30 0x7ff7980b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff797200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7980b4390 0x7ff79815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff797200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980cf070 con 0x7ff79806fb50 2024-09-13T16:54:16.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff797200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980b4be0 con 0x7ff7980b4390 2024-09-13T16:54:16.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff797200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cf330 con 0x7ff79806ff30 2024-09-13T16:54:16.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff79806fb50 0x7ff7980cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff79806fb50 0x7ff7980cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49568/0 (socket says 172.21.15.3:49568) 2024-09-13T16:54:16.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795e00700 1 -- 172.21.15.3:0/1612785693 learned_addr learned my addr 172.21.15.3:0/1612785693 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:16.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795400700 1 --2- 172.21.15.3:0/1612785693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806ff30 0x7ff7980b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff796800700 1 --2- 172.21.15.3:0/1612785693 >> v2:172.21.15.157:3300/0 conn(0x7ff7980b4390 0x7ff79815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795e00700 1 -- 172.21.15.3:0/1612785693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806ff30 msgr2=0x7ff7980b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:16.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795e00700 1 --2- 172.21.15.3:0/1612785693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806ff30 0x7ff7980b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795e00700 1 -- 172.21.15.3:0/1612785693 >> v2:172.21.15.157:3300/0 conn(0x7ff7980b4390 msgr2=0x7ff79815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:16.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795e00700 1 --2- 172.21.15.3:0/1612785693 >> v2:172.21.15.157:3300/0 conn(0x7ff7980b4390 0x7ff79815e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.362+0000 7ff795e00700 1 -- 172.21.15.3:0/1612785693 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980cf610 con 0x7ff79806fb50 2024-09-13T16:54:16.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff795e00700 1 --2- 172.21.15.3:0/1612785693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cfe10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff7980d0b30 tx=0x7ff788009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f7b9b475a4665a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff794a00700 1 -- 172.21.15.3:0/1612785693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788011830 con 0x7ff79806fb50 2024-09-13T16:54:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff794a00700 1 -- 172.21.15.3:0/1612785693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff788011990 con 0x7ff79806fb50 2024-09-13T16:54:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff794a00700 1 -- 172.21.15.3:0/1612785693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788011d20 con 0x7ff79806fb50 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 -- 172.21.15.3:0/1612785693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 msgr2=0x7ff7980cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 --2- 172.21.15.3:0/1612785693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cfe10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff7980d0b30 tx=0x7ff788009bf0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 -- 172.21.15.3:0/1612785693 shutdown_connections 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 --2- 172.21.15.3:0/1612785693 >> v2:172.21.15.157:3300/0 conn(0x7ff7980b4390 0x7ff79815e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 --2- 172.21.15.3:0/1612785693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806ff30 0x7ff7980b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 --2- 172.21.15.3:0/1612785693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cfe10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 -- 172.21.15.3:0/1612785693 >> 172.21.15.3:0/1612785693 conn(0x7ff7980b2910 msgr2=0x7ff7980b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 -- 172.21.15.3:0/1612785693 shutdown_connections 2024-09-13T16:54:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.366+0000 7ff797200700 1 -- 172.21.15.3:0/1612785693 wait complete. 2024-09-13T16:54:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 Processor -- start 2024-09-13T16:54:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 -- start start 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79806ff30 0x7ff79814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7980b4390 0x7ff7980cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79814e2b0 con 0x7ff79806fb50 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980ccef0 con 0x7ff79806ff30 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cd1d0 con 0x7ff7980b4390 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49584/0 (socket says 172.21.15.3:49584) 2024-09-13T16:54:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 -- 172.21.15.3:0/3082067733 learned_addr learned my addr 172.21.15.3:0/3082067733 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff796800700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7980b4390 0x7ff7980cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795400700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79806ff30 0x7ff79814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 -- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7980b4390 msgr2=0x7ff7980cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7980b4390 0x7ff7980cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 -- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79806ff30 msgr2=0x7ff79814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79806ff30 0x7ff79814dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 -- 172.21.15.3:0/3082067733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788009570 con 0x7ff79806fb50 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff795e00700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cc850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff788000c00 tx=0x7ff788004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff78e000700 1 -- 172.21.15.3:0/3082067733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788004300 con 0x7ff79806fb50 2024-09-13T16:54:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff78e000700 1 -- 172.21.15.3:0/3082067733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78801f070 con 0x7ff79806fb50 2024-09-13T16:54:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7980cd4b0 con 0x7ff79806fb50 2024-09-13T16:54:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.370+0000 7ff78e000700 1 -- 172.21.15.3:0/3082067733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880036c0 con 0x7ff79806fb50 2024-09-13T16:54:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.374+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7980cda70 con 0x7ff79806fb50 2024-09-13T16:54:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.374+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7980cfe10 con 0x7ff79806fb50 2024-09-13T16:54:16.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.374+0000 7ff78e000700 1 -- 172.21.15.3:0/3082067733 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff788021030 con 0x7ff79806fb50 2024-09-13T16:54:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.378+0000 7ff78e000700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff778066730 0x7ff778068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.378+0000 7ff78e000700 1 -- 172.21.15.3:0/3082067733 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7ff788085410 con 0x7ff79806fb50 2024-09-13T16:54:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.378+0000 7ff795400700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff778066730 0x7ff778068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.378+0000 7ff795400700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff778066730 0x7ff778068bf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff7980c9ee0 tx=0x7ff784008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.386+0000 7ff78e000700 1 -- 172.21.15.3:0/3082067733 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff788052880 con 0x7ff79806fb50 2024-09-13T16:54:16.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.574+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7980d0350 con 0x7ff778066730 2024-09-13T16:54:16.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.574+0000 7ff78e000700 1 -- 172.21.15.3:0/3082067733 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff784011720 con 0x7ff778066730 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff778066730 msgr2=0x7ff778068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff778066730 0x7ff778068bf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff7980c9ee0 tx=0x7ff784008040 comp rx=0 tx=0).stop 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 msgr2=0x7ff7980cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cc850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff788000c00 tx=0x7ff788004770 comp rx=0 tx=0).stop 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 shutdown_connections 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff778066730 0x7ff778068bf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7980b4390 0x7ff7980cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79806ff30 0x7ff79814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 --2- 172.21.15.3:0/3082067733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff79806fb50 0x7ff7980cc850 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.578+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 >> 172.21.15.3:0/3082067733 conn(0x7ff7980b2910 msgr2=0x7ff798159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.582+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 shutdown_connections 2024-09-13T16:54:16.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.582+0000 7ff797200700 1 -- 172.21.15.3:0/3082067733 wait complete. 2024-09-13T16:54:16.586 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T16:54:16.607 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T16:54:16.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T16:54:16.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.786+0000 7fd87fe00700 1 Processor -- start 2024-09-13T16:54:16.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.786+0000 7fd87fe00700 1 -- start start 2024-09-13T16:54:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.786+0000 7fd87fe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd880151850 0x7fd8800cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.786+0000 7fd87fe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8800d02d0 0x7fd88006f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8800ce650 con 0x7fd880151c30 2024-09-13T16:54:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800ce930 con 0x7fd8800d02d0 2024-09-13T16:54:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800cec10 con 0x7fd880151850 2024-09-13T16:54:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87ea00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd880151850 0x7fd8800cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87ea00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd880151850 0x7fd8800cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.3:42250/0 (socket says 172.21.15.3:42250) 2024-09-13T16:54:16.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87ea00700 1 -- 172.21.15.3:0/3621683419 learned_addr learned my addr 172.21.15.3:0/3621683419 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:16.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87f400700 1 --2- 172.21.15.3:0/3621683419 >> v2:172.21.15.3:3300/0 conn(0x7fd8800d02d0 0x7fd88006f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87ea00700 1 -- 172.21.15.3:0/3621683419 >> v2:172.21.15.3:3300/0 conn(0x7fd8800d02d0 msgr2=0x7fd88006f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:16.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87ea00700 1 --2- 172.21.15.3:0/3621683419 >> v2:172.21.15.3:3300/0 conn(0x7fd8800d02d0 0x7fd88006f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87ea00700 1 -- 172.21.15.3:0/3621683419 >> v1:172.21.15.3:6789/0 conn(0x7fd880151c30 legacy=0x7fd8800cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87ea00700 1 -- 172.21.15.3:0/3621683419 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8800ceef0 con 0x7fd880151850 2024-09-13T16:54:16.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87ea00700 1 --2- 172.21.15.3:0/3621683419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 0x7fd8800cdd70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd8800b3380 tx=0x7fd870009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769d8e9f69889a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87d600700 1 -- 172.21.15.3:0/3621683419 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87000b8a0 con 0x7fd880151850 2024-09-13T16:54:16.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87d600700 1 -- 172.21.15.3:0/3621683419 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87000ba00 con 0x7fd880151850 2024-09-13T16:54:16.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87d600700 1 -- 172.21.15.3:0/3621683419 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87000bd90 con 0x7fd880151850 2024-09-13T16:54:16.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 -- 172.21.15.3:0/3621683419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 msgr2=0x7fd8800cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 --2- 172.21.15.3:0/3621683419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 0x7fd8800cdd70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd8800b3380 tx=0x7fd870009bf0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 -- 172.21.15.3:0/3621683419 shutdown_connections 2024-09-13T16:54:16.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 --2- 172.21.15.3:0/3621683419 >> v2:172.21.15.3:3300/0 conn(0x7fd8800d02d0 0x7fd88006f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 --2- 172.21.15.3:0/3621683419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 0x7fd8800cdd70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 -- 172.21.15.3:0/3621683419 >> 172.21.15.3:0/3621683419 conn(0x7fd8800b2830 msgr2=0x7fd8800b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:16.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 -- 172.21.15.3:0/3621683419 shutdown_connections 2024-09-13T16:54:16.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.790+0000 7fd87fe00700 1 -- 172.21.15.3:0/3621683419 wait complete. 2024-09-13T16:54:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87fe00700 1 Processor -- start 2024-09-13T16:54:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87fe00700 1 -- start start 2024-09-13T16:54:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8800d02d0 0x7fd88014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 0x7fd88015d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87fe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd880151c30 0x7fd880153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87fe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88014e280 con 0x7fd8800d02d0 2024-09-13T16:54:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88015c700 con 0x7fd880151850 2024-09-13T16:54:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87fe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88015c9e0 con 0x7fd880151c30 2024-09-13T16:54:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8800d02d0 0x7fd88014dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8800d02d0 0x7fd88014dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49596/0 (socket says 172.21.15.3:49596) 2024-09-13T16:54:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87ea00700 1 -- 172.21.15.3:0/2561344719 learned_addr learned my addr 172.21.15.3:0/2561344719 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.794+0000 7fd87e000700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 0x7fd88015d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87f400700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd880151c30 0x7fd880153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87ea00700 1 -- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd880151c30 msgr2=0x7fd880153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87ea00700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd880151c30 0x7fd880153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87ea00700 1 -- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 msgr2=0x7fd88015d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87ea00700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 0x7fd88015d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87ea00700 1 -- 172.21.15.3:0/2561344719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870009570 con 0x7fd8800d02d0 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87e000700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd880151850 0x7fd88015d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87ea00700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8800d02d0 0x7fd88014dd40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd8800cf210 tx=0x7fd870004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8700036c0 con 0x7fd8800d02d0 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87001f070 con 0x7fd8800d02d0 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88015ccc0 con 0x7fd8800d02d0 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870003a80 con 0x7fd8800d02d0 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88015d280 con 0x7fd8800d02d0 2024-09-13T16:54:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.798+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd88015bc00 con 0x7fd8800d02d0 2024-09-13T16:54:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd870021030 con 0x7fd8800d02d0 2024-09-13T16:54:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd876a00700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd860066730 0x7fd860068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd870085bf0 con 0x7fd8800d02d0 2024-09-13T16:54:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd876a00700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7fd86006cb70 0x7fd86006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 --> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd86006f680 con 0x7fd86006cb70 2024-09-13T16:54:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd870085f60 con 0x7fd8800d02d0 2024-09-13T16:54:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd87e000700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd860066730 0x7fd860068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd87f400700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7fd86006cb70 0x7fd86006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.802+0000 7fd87f400700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7fd86006cb70 0x7fd86006efb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.806+0000 7fd87e000700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd860066730 0x7fd860068bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd88015f420 tx=0x7fd86c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.806+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 <== osd.1 v2:172.21.15.3:6800/4234138253 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd86006f680 con 0x7fd86006cb70 2024-09-13T16:54:16.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.822+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 --> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd88006b2f0 con 0x7fd86006cb70 2024-09-13T16:54:16.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.822+0000 7fd876a00700 1 -- 172.21.15.3:0/2561344719 <== osd.1 v2:172.21.15.3:6800/4234138253 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd88006b2f0 con 0x7fd86006cb70 2024-09-13T16:54:16.828 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:54:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T16:54:16.828 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:54:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7fd86006cb70 msgr2=0x7fd86006efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.3:6800/4234138253,v1:172.21.15.3:6801/4234138253] conn(0x7fd86006cb70 0x7fd86006efb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd860066730 msgr2=0x7fd860068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd860066730 0x7fd860068bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd88015f420 tx=0x7fd86c009040 comp rx=0 tx=0).stop 2024-09-13T16:54:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8800d02d0 msgr2=0x7fd88014dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 --2- 172.21.15.3:0/2561344719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8800d02d0 0x7fd88014dd40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd8800cf210 tx=0x7fd870004770 comp rx=0 tx=0).stop 2024-09-13T16:54:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87f400700 1 -- 172.21.15.3:0/2561344719 reap_dead start 2024-09-13T16:54:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 shutdown_connections 2024-09-13T16:54:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 >> 172.21.15.3:0/2561344719 conn(0x7fd8800b2830 msgr2=0x7fd8800b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 shutdown_connections 2024-09-13T16:54:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:16.826+0000 7fd87fe00700 1 -- 172.21.15.3:0/2561344719 wait complete. 2024-09-13T16:54:16.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T16:54:17.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.030+0000 7f153ec00700 1 Processor -- start 2024-09-13T16:54:17.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.030+0000 7f153ec00700 1 -- start start 2024-09-13T16:54:17.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.030+0000 7f153ec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f153806ccf0 0x7f1538165820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.030+0000 7f153ec00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15380b4100 con 0x7f15381443b0 2024-09-13T16:54:17.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.030+0000 7f153ec00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15380ce8e0 con 0x7f1538144790 2024-09-13T16:54:17.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.030+0000 7f153ec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15380ccd30 con 0x7f153806ccf0 2024-09-13T16:54:17.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f153d800700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f15381443b0 0x7f15380b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56594/0 (socket says 172.21.15.3:56594) 2024-09-13T16:54:17.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f153d800700 1 -- 172.21.15.3:0/3792367189 learned_addr learned my addr 172.21.15.3:0/3792367189 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:17.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f153e200700 1 --2- 172.21.15.3:0/3792367189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f153806ccf0 0x7f1538165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f1537e00700 1 -- 172.21.15.3:0/3792367189 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943609423 0 0) 0x7f15380ce8e0 con 0x7f1538144790 2024-09-13T16:54:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f1537e00700 1 -- 172.21.15.3:0/3792367189 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15200035d0 con 0x7f1538144790 2024-09-13T16:54:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f1537e00700 1 -- 172.21.15.3:0/3792367189 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994842672 0 0) 0x7f15380b4100 con 0x7f15381443b0 2024-09-13T16:54:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f1537e00700 1 -- 172.21.15.3:0/3792367189 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15380ce8e0 con 0x7f15381443b0 2024-09-13T16:54:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f153e200700 1 -- 172.21.15.3:0/3792367189 >> v1:172.21.15.157:6789/0 conn(0x7f1538144790 legacy=0x7f153806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f153e200700 1 -- 172.21.15.3:0/3792367189 >> v1:172.21.15.3:6789/0 conn(0x7f15381443b0 legacy=0x7f15380b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f153e200700 1 -- 172.21.15.3:0/3792367189 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15380cd010 con 0x7f153806ccf0 2024-09-13T16:54:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f153e200700 1 --2- 172.21.15.3:0/3792367189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f153806ccf0 0x7f1538165820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1530009dc0 tx=0x7f153000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83aa2d4ba4610cec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f1537e00700 1 -- 172.21.15.3:0/3792367189 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000e070 con 0x7f153806ccf0 2024-09-13T16:54:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f1537e00700 1 -- 172.21.15.3:0/3792367189 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153000bb20 con 0x7f153806ccf0 2024-09-13T16:54:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.034+0000 7f1537e00700 1 -- 172.21.15.3:0/3792367189 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530012430 con 0x7f153806ccf0 2024-09-13T16:54:17.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.038+0000 7f153ec00700 1 -- 172.21.15.3:0/3792367189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f153806ccf0 msgr2=0x7f1538165820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.038+0000 7f153ec00700 1 --2- 172.21.15.3:0/3792367189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f153806ccf0 0x7f1538165820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1530009dc0 tx=0x7f153000b330 comp rx=0 tx=0).stop 2024-09-13T16:54:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.038+0000 7f153ec00700 1 -- 172.21.15.3:0/3792367189 shutdown_connections 2024-09-13T16:54:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.038+0000 7f153ec00700 1 --2- 172.21.15.3:0/3792367189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f153806ccf0 0x7f1538165820 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.038+0000 7f153ec00700 1 -- 172.21.15.3:0/3792367189 >> 172.21.15.3:0/3792367189 conn(0x7f15380b0cf0 msgr2=0x7f15380b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.038+0000 7f153ec00700 1 -- 172.21.15.3:0/3792367189 shutdown_connections 2024-09-13T16:54:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.038+0000 7f153ec00700 1 -- 172.21.15.3:0/3792367189 wait complete. 2024-09-13T16:54:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ec00700 1 Processor -- start 2024-09-13T16:54:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ec00700 1 -- start start 2024-09-13T16:54:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f153806ccf0 0x7f15381ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ec00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15381443b0 0x7f15380ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ec00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15381f2cf0 0x7f153806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ec00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381efd20 con 0x7f15381f2cf0 2024-09-13T16:54:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15380ada00 con 0x7f153806ccf0 2024-09-13T16:54:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ec00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1538071ed0 con 0x7f15381443b0 2024-09-13T16:54:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15381f2cf0 0x7f153806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15381f2cf0 0x7f153806fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49604/0 (socket says 172.21.15.3:49604) 2024-09-13T16:54:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 -- 172.21.15.3:0/938407875 learned_addr learned my addr 172.21.15.3:0/938407875 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153ce00700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15381443b0 0x7f15380ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153d800700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f153806ccf0 0x7f15381ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 -- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15381443b0 msgr2=0x7f15380ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15381443b0 0x7f15380ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 -- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f153806ccf0 msgr2=0x7f15381ef7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f153806ccf0 0x7f15381ef7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 -- 172.21.15.3:0/938407875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1530009a70 con 0x7f15381f2cf0 2024-09-13T16:54:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153d800700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f153806ccf0 0x7f15381ef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.042+0000 7f153e200700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15381f2cf0 0x7f153806fea0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f153000b370 tx=0x7f1530004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.046+0000 7f1535600700 1 -- 172.21.15.3:0/938407875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153001b810 con 0x7f15381f2cf0 2024-09-13T16:54:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.046+0000 7f1535600700 1 -- 172.21.15.3:0/938407875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530020070 con 0x7f15381f2cf0 2024-09-13T16:54:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.046+0000 7f1535600700 1 -- 172.21.15.3:0/938407875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153001bbf0 con 0x7f15381f2cf0 2024-09-13T16:54:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.046+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15380c3df0 con 0x7f15381f2cf0 2024-09-13T16:54:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.046+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15380ac9e0 con 0x7f15381f2cf0 2024-09-13T16:54:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.046+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153806c5e0 con 0x7f15381f2cf0 2024-09-13T16:54:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.050+0000 7f1535600700 1 -- 172.21.15.3:0/938407875 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f153001bd50 con 0x7f15381f2cf0 2024-09-13T16:54:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.050+0000 7f1535600700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1518066730 0x7f1518068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.050+0000 7f1535600700 1 -- 172.21.15.3:0/938407875 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f1530022070 con 0x7f15381f2cf0 2024-09-13T16:54:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.050+0000 7f153d800700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1518066730 0x7f1518068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.050+0000 7f153d800700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1518066730 0x7f1518068bf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f152c0011e0 tx=0x7f152c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.058+0000 7f1535600700 1 -- 172.21.15.3:0/938407875 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15300537c0 con 0x7f15381f2cf0 2024-09-13T16:54:17.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.242+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f15380cd8b0 con 0x7f1518066730 2024-09-13T16:54:17.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.242+0000 7f1535600700 1 -- 172.21.15.3:0/938407875 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f152c0014f0 con 0x7f1518066730 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1518066730 msgr2=0x7f1518068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1518066730 0x7f1518068bf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f152c0011e0 tx=0x7f152c009040 comp rx=0 tx=0).stop 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15381f2cf0 msgr2=0x7f153806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15381f2cf0 0x7f153806fea0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f153000b370 tx=0x7f1530004af0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 shutdown_connections 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f1518066730 0x7f1518068bf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15381f2cf0 0x7f153806fea0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15381443b0 0x7f15380ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 --2- 172.21.15.3:0/938407875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f153806ccf0 0x7f15381ef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 >> 172.21.15.3:0/938407875 conn(0x7f15380b0cf0 msgr2=0x7f15380b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:17.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 shutdown_connections 2024-09-13T16:54:17.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.246+0000 7f153ec00700 1 -- 172.21.15.3:0/938407875 wait complete. 2024-09-13T16:54:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T16:54:17.269 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T16:54:17.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T16:54:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.454+0000 7f708ee00700 1 Processor -- start 2024-09-13T16:54:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.454+0000 7f708ee00700 1 -- start start 2024-09-13T16:54:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.454+0000 7f708ee00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 0x7f708806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.454+0000 7f708ee00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70880b1ec0 con 0x7f708806d3b0 2024-09-13T16:54:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.454+0000 7f708ee00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7088070830 con 0x7f7088160e50 2024-09-13T16:54:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.454+0000 7f708ee00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70880710d0 con 0x7f7088071cf0 2024-09-13T16:54:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708e400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f708806d3b0 0x7f70880b2350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56604/0 (socket says 172.21.15.3:56604) 2024-09-13T16:54:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708e400700 1 -- 172.21.15.3:0/2176131275 learned_addr learned my addr 172.21.15.3:0/2176131275 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708d000700 1 --2- 172.21.15.3:0/2176131275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 0x7f708806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f7087e00700 1 -- 172.21.15.3:0/2176131275 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212609570 0 0) 0x7f7088070830 con 0x7f7088160e50 2024-09-13T16:54:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f7087e00700 1 -- 172.21.15.3:0/2176131275 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70700035d0 con 0x7f7088160e50 2024-09-13T16:54:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f7087e00700 1 -- 172.21.15.3:0/2176131275 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111381411 0 0) 0x7f70880b1ec0 con 0x7f708806d3b0 2024-09-13T16:54:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f7087e00700 1 -- 172.21.15.3:0/2176131275 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7088070830 con 0x7f708806d3b0 2024-09-13T16:54:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708d000700 1 -- 172.21.15.3:0/2176131275 >> v1:172.21.15.157:6789/0 conn(0x7f7088160e50 legacy=0x7f70880cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708d000700 1 -- 172.21.15.3:0/2176131275 >> v1:172.21.15.3:6789/0 conn(0x7f708806d3b0 legacy=0x7f70880b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708d000700 1 -- 172.21.15.3:0/2176131275 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7088070b10 con 0x7f7088071cf0 2024-09-13T16:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708d000700 1 --2- 172.21.15.3:0/2176131275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 0x7f708806fec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7074009880 tx=0x7f7074009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec075917fce3100 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f7087e00700 1 -- 172.21.15.3:0/2176131275 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074011830 con 0x7f7088071cf0 2024-09-13T16:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f7087e00700 1 -- 172.21.15.3:0/2176131275 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7074011990 con 0x7f7088071cf0 2024-09-13T16:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f7087e00700 1 -- 172.21.15.3:0/2176131275 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074011d20 con 0x7f7088071cf0 2024-09-13T16:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708ee00700 1 -- 172.21.15.3:0/2176131275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 msgr2=0x7f708806fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.458+0000 7f708ee00700 1 --2- 172.21.15.3:0/2176131275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 0x7f708806fec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7074009880 tx=0x7f7074009b90 comp rx=0 tx=0).stop 2024-09-13T16:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 -- 172.21.15.3:0/2176131275 shutdown_connections 2024-09-13T16:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 --2- 172.21.15.3:0/2176131275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 0x7f708806fec0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 -- 172.21.15.3:0/2176131275 >> 172.21.15.3:0/2176131275 conn(0x7f708806b1e0 msgr2=0x7f708806b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 -- 172.21.15.3:0/2176131275 shutdown_connections 2024-09-13T16:54:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 -- 172.21.15.3:0/2176131275 wait complete. 2024-09-13T16:54:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 Processor -- start 2024-09-13T16:54:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 -- start start 2024-09-13T16:54:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f708806d3b0 0x7f70881636f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 0x7f70881549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7088160e50 0x7f70880ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70881fadd0 con 0x7f7088160e50 2024-09-13T16:54:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088163260 con 0x7f708806d3b0 2024-09-13T16:54:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708ee00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70881faaf0 con 0x7f7088071cf0 2024-09-13T16:54:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.462+0000 7f708e400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7088160e50 0x7f70880ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708e400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7088160e50 0x7f70880ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49618/0 (socket says 172.21.15.3:49618) 2024-09-13T16:54:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708e400700 1 -- 172.21.15.3:0/1302639852 learned_addr learned my addr 172.21.15.3:0/1302639852 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708da00700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f708806d3b0 0x7f70881636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708d000700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 0x7f70881549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708e400700 1 -- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 msgr2=0x7f70881549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708e400700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7088071cf0 0x7f70881549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708e400700 1 -- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f708806d3b0 msgr2=0x7f70881636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708e400700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f708806d3b0 0x7f70881636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708e400700 1 -- 172.21.15.3:0/1302639852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70740094b0 con 0x7f7088160e50 2024-09-13T16:54:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708e400700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7088160e50 0x7f70880ccf00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f708000cb40 tx=0x7f708000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080007500 con 0x7f7088160e50 2024-09-13T16:54:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080007660 con 0x7f7088160e50 2024-09-13T16:54:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7088162c10 con 0x7f7088160e50 2024-09-13T16:54:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800079f0 con 0x7f7088160e50 2024-09-13T16:54:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7088154f10 con 0x7f7088160e50 2024-09-13T16:54:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.466+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7088070400 con 0x7f7088160e50 2024-09-13T16:54:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.470+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f708001b3b0 con 0x7f7088160e50 2024-09-13T16:54:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.470+0000 7f7085600700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7068066760 0x7f7068068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.470+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f7080084180 con 0x7f7088160e50 2024-09-13T16:54:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.474+0000 7f7085600700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f706806cba0 0x7f706806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.474+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 --> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f706806f6b0 con 0x7f706806cba0 2024-09-13T16:54:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.474+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70800844f0 con 0x7f7088160e50 2024-09-13T16:54:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.474+0000 7f708da00700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7068066760 0x7f7068068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.474+0000 7f708d000700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f706806cba0 0x7f706806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.474+0000 7f708d000700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f706806cba0 0x7f706806efe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.474+0000 7f708da00700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7068066760 0x7f7068068c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f707c0011e0 tx=0x7f707c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.474+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 <== osd.2 v2:172.21.15.3:6808/4113809917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f706806f6b0 con 0x7f706806cba0 2024-09-13T16:54:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 --> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7088162a70 con 0x7f706806cba0 2024-09-13T16:54:17.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f7085600700 1 -- 172.21.15.3:0/1302639852 <== osd.2 v2:172.21.15.3:6808/4113809917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7088162a70 con 0x7f706806cba0 2024-09-13T16:54:17.497 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:54:17.497 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T16:54:17.497 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:54:17.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f706806cba0 msgr2=0x7f706806efe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.3:6808/4113809917,v1:172.21.15.3:6809/4113809917] conn(0x7f706806cba0 0x7f706806efe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7068066760 msgr2=0x7f7068068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f7068066760 0x7f7068068c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f707c0011e0 tx=0x7f707c009040 comp rx=0 tx=0).stop 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7088160e50 msgr2=0x7f70880ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 --2- 172.21.15.3:0/1302639852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7088160e50 0x7f70880ccf00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f708000cb40 tx=0x7f708000cb70 comp rx=0 tx=0).stop 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708e400700 1 -- 172.21.15.3:0/1302639852 reap_dead start 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 shutdown_connections 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 >> 172.21.15.3:0/1302639852 conn(0x7f708806b1e0 msgr2=0x7f708815b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 shutdown_connections 2024-09-13T16:54:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.494+0000 7f708ee00700 1 -- 172.21.15.3:0/1302639852 wait complete. 2024-09-13T16:54:17.524 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T16:54:17.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.690+0000 7f3d0e200700 1 Processor -- start 2024-09-13T16:54:17.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.690+0000 7f3d0e200700 1 -- start start 2024-09-13T16:54:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d0e200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d08153e80 0x7f3d080cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d0e200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0815db60 0x7f3d080ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d0e200700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0806c820 con 0x7f3d080b4e70 2024-09-13T16:54:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d0e200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d080b5300 con 0x7f3d08153e80 2024-09-13T16:54:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d0e200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0806dde0 con 0x7f3d0815db60 2024-09-13T16:54:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d07e00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3d080b4e70 0x7f3d0815d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56612/0 (socket says 172.21.15.3:56612) 2024-09-13T16:54:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d07e00700 1 -- 172.21.15.3:0/3812859573 learned_addr learned my addr 172.21.15.3:0/3812859573 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d0ce00700 1 --2- 172.21.15.3:0/3812859573 >> v2:172.21.15.157:3300/0 conn(0x7f3d08153e80 0x7f3d080cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d0d800700 1 --2- 172.21.15.3:0/3812859573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0815db60 0x7f3d080ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.694+0000 7f3d07400700 1 -- 172.21.15.3:0/3812859573 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082928625 0 0) 0x7f3d0806c820 con 0x7f3d080b4e70 2024-09-13T16:54:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d07400700 1 -- 172.21.15.3:0/3812859573 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf00057e0 con 0x7f3d080b4e70 2024-09-13T16:54:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0ce00700 1 -- 172.21.15.3:0/3812859573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0815db60 msgr2=0x7f3d080ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0ce00700 1 --2- 172.21.15.3:0/3812859573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0815db60 0x7f3d080ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0ce00700 1 -- 172.21.15.3:0/3812859573 >> v1:172.21.15.3:6789/0 conn(0x7f3d080b4e70 legacy=0x7f3d0815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0ce00700 1 -- 172.21.15.3:0/3812859573 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0806d820 con 0x7f3d08153e80 2024-09-13T16:54:17.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0ce00700 1 --2- 172.21.15.3:0/3812859573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d08153e80 0x7f3d080cc780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009850 tx=0x7f3cfc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b252df74f4503aed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d07400700 1 -- 172.21.15.3:0/3812859573 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc011830 con 0x7f3d08153e80 2024-09-13T16:54:17.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d07400700 1 -- 172.21.15.3:0/3812859573 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc011990 con 0x7f3d08153e80 2024-09-13T16:54:17.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d07400700 1 -- 172.21.15.3:0/3812859573 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc011d40 con 0x7f3d08153e80 2024-09-13T16:54:17.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0e200700 1 -- 172.21.15.3:0/3812859573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d08153e80 msgr2=0x7f3d080cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0e200700 1 --2- 172.21.15.3:0/3812859573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d08153e80 0x7f3d080cc780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009850 tx=0x7f3cfc009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0e200700 1 -- 172.21.15.3:0/3812859573 shutdown_connections 2024-09-13T16:54:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0e200700 1 --2- 172.21.15.3:0/3812859573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0815db60 0x7f3d080ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0e200700 1 --2- 172.21.15.3:0/3812859573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d08153e80 0x7f3d080cc780 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0e200700 1 -- 172.21.15.3:0/3812859573 >> 172.21.15.3:0/3812859573 conn(0x7f3d080710b0 msgr2=0x7f3d080714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0e200700 1 -- 172.21.15.3:0/3812859573 shutdown_connections 2024-09-13T16:54:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.698+0000 7f3d0e200700 1 -- 172.21.15.3:0/3812859573 wait complete. 2024-09-13T16:54:17.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.702+0000 7f3d0e200700 1 Processor -- start 2024-09-13T16:54:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.702+0000 7f3d0e200700 1 -- start start 2024-09-13T16:54:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d080b4e70 0x7f3d081f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0e200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d08153e80 0x7f3d08151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0e200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d0815db60 0x7f3d08152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0e200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d081f22d0 con 0x7f3d0815db60 2024-09-13T16:54:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08151680 con 0x7f3d080b4e70 2024-09-13T16:54:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0e200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08151960 con 0x7f3d08153e80 2024-09-13T16:54:17.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d0815db60 0x7f3d08152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d0815db60 0x7f3d08152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49630/0 (socket says 172.21.15.3:49630) 2024-09-13T16:54:17.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 -- 172.21.15.3:0/4024955267 learned_addr learned my addr 172.21.15.3:0/4024955267 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:17.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d07e00700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d08153e80 0x7f3d08151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0ce00700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d080b4e70 0x7f3d081f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 -- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d08153e80 msgr2=0x7f3d08151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d08153e80 0x7f3d08151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 -- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d080b4e70 msgr2=0x7f3d081f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d080b4e70 0x7f3d081f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 -- 172.21.15.3:0/4024955267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0094b0 con 0x7f3d0815db60 2024-09-13T16:54:17.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d07e00700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d08153e80 0x7f3d08151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:17.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0d800700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d0815db60 0x7f3d08152290 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000efc0 tx=0x7f3d0000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d04c00700 1 -- 172.21.15.3:0/4024955267 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00009ea0 con 0x7f3d0815db60 2024-09-13T16:54:17.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d04c00700 1 -- 172.21.15.3:0/4024955267 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0000f040 con 0x7f3d0815db60 2024-09-13T16:54:17.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d080a85d0 con 0x7f3d0815db60 2024-09-13T16:54:17.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d04c00700 1 -- 172.21.15.3:0/4024955267 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00014480 con 0x7f3d0815db60 2024-09-13T16:54:17.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.706+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d080a8b90 con 0x7f3d0815db60 2024-09-13T16:54:17.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.710+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0814f0b0 con 0x7f3d0815db60 2024-09-13T16:54:17.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.710+0000 7f3d04c00700 1 -- 172.21.15.3:0/4024955267 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d000145e0 con 0x7f3d0815db60 2024-09-13T16:54:17.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.714+0000 7f3d04c00700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3ce8066730 0x7f3ce8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.714+0000 7f3d04c00700 1 -- 172.21.15.3:0/4024955267 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f3d00084b70 con 0x7f3d0815db60 2024-09-13T16:54:17.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.714+0000 7f3d0ce00700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3ce8066730 0x7f3ce8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.714+0000 7f3d0ce00700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3ce8066730 0x7f3ce8068bf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0090b0 tx=0x7f3cfc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.718+0000 7f3d04c00700 1 -- 172.21.15.3:0/4024955267 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3d00051fe0 con 0x7f3d0815db60 2024-09-13T16:54:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.902+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d08154260 con 0x7f3ce8066730 2024-09-13T16:54:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.902+0000 7f3d04c00700 1 -- 172.21.15.3:0/4024955267 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3cfc011b00 con 0x7f3ce8066730 2024-09-13T16:54:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3ce8066730 msgr2=0x7f3ce8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3ce8066730 0x7f3ce8068bf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0090b0 tx=0x7f3cfc008030 comp rx=0 tx=0).stop 2024-09-13T16:54:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d0815db60 msgr2=0x7f3d08152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d0815db60 0x7f3d08152290 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000efc0 tx=0x7f3d0000c3a0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 shutdown_connections 2024-09-13T16:54:17.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3ce8066730 0x7f3ce8068bf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d0815db60 0x7f3d08152290 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d08153e80 0x7f3d08151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 --2- 172.21.15.3:0/4024955267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d080b4e70 0x7f3d081f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 >> 172.21.15.3:0/4024955267 conn(0x7f3d080710b0 msgr2=0x7f3d0806e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:17.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 shutdown_connections 2024-09-13T16:54:17.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:17.906+0000 7f3d0e200700 1 -- 172.21.15.3:0/4024955267 wait complete. 2024-09-13T16:54:17.912 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T16:54:17.932 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T16:54:17.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T16:54:18.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.114+0000 7f8703e00700 1 Processor -- start 2024-09-13T16:54:18.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.114+0000 7f8703e00700 1 -- start start 2024-09-13T16:54:18.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.114+0000 7f8703e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87041615b0 0x7f8704161990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.114+0000 7f8703e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8704161ed0 0x7f87040ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.114+0000 7f8703e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87040ced80 con 0x7f87040ce570 2024-09-13T16:54:18.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.114+0000 7f8703e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87040cf060 con 0x7f87041615b0 2024-09-13T16:54:18.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.114+0000 7f8703e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87040b3bd0 con 0x7f8704161ed0 2024-09-13T16:54:18.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f87040ce570 0x7f87040b46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56616/0 (socket says 172.21.15.3:56616) 2024-09-13T16:54:18.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703400700 1 -- 172.21.15.3:0/3258564488 learned_addr learned my addr 172.21.15.3:0/3258564488 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:18.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702a00700 1 --2- 172.21.15.3:0/3258564488 >> v2:172.21.15.3:3300/0 conn(0x7f87041615b0 0x7f8704161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f86fbe00700 1 --2- 172.21.15.3:0/3258564488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8704161ed0 0x7f87040ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702000700 1 -- 172.21.15.3:0/3258564488 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361924441 0 0) 0x7f87040ced80 con 0x7f87040ce570 2024-09-13T16:54:18.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702000700 1 -- 172.21.15.3:0/3258564488 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e80035d0 con 0x7f87040ce570 2024-09-13T16:54:18.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702000700 1 -- 172.21.15.3:0/3258564488 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2279761443 0 0) 0x7f86e80035d0 con 0x7f87040ce570 2024-09-13T16:54:18.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702a00700 1 -- 172.21.15.3:0/3258564488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8704161ed0 msgr2=0x7f87040ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:18.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702a00700 1 --2- 172.21.15.3:0/3258564488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8704161ed0 0x7f87040ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702a00700 1 -- 172.21.15.3:0/3258564488 >> v1:172.21.15.3:6789/0 conn(0x7f87040ce570 legacy=0x7f87040b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702a00700 1 -- 172.21.15.3:0/3258564488 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87040b3eb0 con 0x7f87041615b0 2024-09-13T16:54:18.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f86fbe00700 1 --2- 172.21.15.3:0/3258564488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8704161ed0 0x7f87040ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:18.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702a00700 1 --2- 172.21.15.3:0/3258564488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87041615b0 0x7f8704161990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009850 tx=0x7f86f4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfee80bbc6da84d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702000700 1 -- 172.21.15.3:0/3258564488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0019f0 con 0x7f87041615b0 2024-09-13T16:54:18.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702000700 1 -- 172.21.15.3:0/3258564488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4004030 con 0x7f87041615b0 2024-09-13T16:54:18.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8702000700 1 -- 172.21.15.3:0/3258564488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40043a0 con 0x7f87041615b0 2024-09-13T16:54:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703e00700 1 -- 172.21.15.3:0/3258564488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87041615b0 msgr2=0x7f8704161990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703e00700 1 --2- 172.21.15.3:0/3258564488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87041615b0 0x7f8704161990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009850 tx=0x7f86f4009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703e00700 1 -- 172.21.15.3:0/3258564488 shutdown_connections 2024-09-13T16:54:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703e00700 1 --2- 172.21.15.3:0/3258564488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8704161ed0 0x7f87040ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703e00700 1 --2- 172.21.15.3:0/3258564488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87041615b0 0x7f8704161990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703e00700 1 -- 172.21.15.3:0/3258564488 >> 172.21.15.3:0/3258564488 conn(0x7f87040b2830 msgr2=0x7f87040b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.118+0000 7f8703e00700 1 -- 172.21.15.3:0/3258564488 shutdown_connections 2024-09-13T16:54:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.122+0000 7f8703e00700 1 -- 172.21.15.3:0/3258564488 wait complete. 2024-09-13T16:54:18.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.122+0000 7f8703e00700 1 Processor -- start 2024-09-13T16:54:18.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.122+0000 7f8703e00700 1 -- start start 2024-09-13T16:54:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87040ce570 0x7f87040c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87041615b0 0x7f8704159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8704161ed0 0x7f870415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870415f290 con 0x7f87040ce570 2024-09-13T16:54:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87040c6e50 con 0x7f8704161ed0 2024-09-13T16:54:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87040c7130 con 0x7f87041615b0 2024-09-13T16:54:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87040ce570 0x7f87040c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87040ce570 0x7f87040c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49638/0 (socket says 172.21.15.3:49638) 2024-09-13T16:54:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 -- 172.21.15.3:0/3300413808 learned_addr learned my addr 172.21.15.3:0/3300413808 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703400700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8704161ed0 0x7f870415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f86fbe00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87041615b0 0x7f8704159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 -- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87041615b0 msgr2=0x7f8704159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87041615b0 0x7f8704159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 -- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8704161ed0 msgr2=0x7f870415a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8704161ed0 0x7f870415a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 -- 172.21.15.3:0/3300413808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0094b0 con 0x7f87040ce570 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703400700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8704161ed0 0x7f870415a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8702a00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87040ce570 0x7f87040c80d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f86f4000c00 tx=0x7f86f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4011630 con 0x7f87040ce570 2024-09-13T16:54:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f401f070 con 0x7f87040ce570 2024-09-13T16:54:18.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f40094b0 con 0x7f87040ce570 2024-09-13T16:54:18.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40119d0 con 0x7f87040ce570 2024-09-13T16:54:18.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.126+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870415ef20 con 0x7f87040ce570 2024-09-13T16:54:18.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.130+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87040cf970 con 0x7f87040ce570 2024-09-13T16:54:18.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.130+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86f4011b30 con 0x7f87040ce570 2024-09-13T16:54:18.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.130+0000 7f86faa00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86e0066730 0x7f86e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f86f4021070 con 0x7f87040ce570 2024-09-13T16:54:18.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f86faa00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f86e006cb70 0x7f86e006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 --> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86e006f680 con 0x7f86e006cb70 2024-09-13T16:54:18.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86f4087e20 con 0x7f87040ce570 2024-09-13T16:54:18.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f86fbe00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86e0066730 0x7f86e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f8703400700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f86e006cb70 0x7f86e006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f8703400700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f86e006cb70 0x7f86e006efb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f86fbe00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86e0066730 0x7f86e0068bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8704161510 tx=0x7f86ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.134+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 <== osd.3 v2:172.21.15.3:6813/1788023973 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f86e006f680 con 0x7f86e006cb70 2024-09-13T16:54:18.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 --> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8704162340 con 0x7f86e006cb70 2024-09-13T16:54:18.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f86faa00700 1 -- 172.21.15.3:0/3300413808 <== osd.3 v2:172.21.15.3:6813/1788023973 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f86fc0070e0 con 0x7f86e006cb70 2024-09-13T16:54:18.158 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:54:18.159 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T16:54:18.159 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f86e006cb70 msgr2=0x7f86e006efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.3:6813/1788023973,v1:172.21.15.3:6816/1788023973] conn(0x7f86e006cb70 0x7f86e006efb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86e0066730 msgr2=0x7f86e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f86e0066730 0x7f86e0068bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8704161510 tx=0x7f86ec008040 comp rx=0 tx=0).stop 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87040ce570 msgr2=0x7f87040c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 --2- 172.21.15.3:0/3300413808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87040ce570 0x7f87040c80d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f86f4000c00 tx=0x7f86f4004770 comp rx=0 tx=0).stop 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703400700 1 -- 172.21.15.3:0/3300413808 reap_dead start 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 shutdown_connections 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 >> 172.21.15.3:0/3300413808 conn(0x7f87040b2830 msgr2=0x7f87040b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 shutdown_connections 2024-09-13T16:54:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.154+0000 7f8703e00700 1 -- 172.21.15.3:0/3300413808 wait complete. 2024-09-13T16:54:18.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T16:54:18.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.358+0000 7fd532c00700 1 Processor -- start 2024-09-13T16:54:18.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.358+0000 7fd532c00700 1 -- start start 2024-09-13T16:54:18.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd532c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd532c00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd52c0ce700 0x7fd52c06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd532c00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c0b61e0 con 0x7fd52c15a850 2024-09-13T16:54:18.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd532c00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52c0cebb0 con 0x7fd52c0ce700 2024-09-13T16:54:18.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd532c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c0b55d0 con 0x7fd52c15ac30 2024-09-13T16:54:18.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd530e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd530e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c0ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:45090/0 (socket says 172.21.15.3:45090) 2024-09-13T16:54:18.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd530e00700 1 -- 172.21.15.3:0/1341227518 learned_addr learned my addr 172.21.15.3:0/1341227518 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:18.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd531800700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd52c15a850 0x7fd52c0b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.3:46092/0 (socket says 172.21.15.3:46092) 2024-09-13T16:54:18.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd532200700 1 --2- 172.21.15.3:0/1341227518 >> v2:172.21.15.157:3300/0 conn(0x7fd52c0ce700 0x7fd52c06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd530e00700 1 -- 172.21.15.3:0/1341227518 >> v2:172.21.15.157:3300/0 conn(0x7fd52c0ce700 msgr2=0x7fd52c06d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:18.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd530e00700 1 --2- 172.21.15.3:0/1341227518 >> v2:172.21.15.157:3300/0 conn(0x7fd52c0ce700 0x7fd52c06d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd530e00700 1 -- 172.21.15.3:0/1341227518 >> v1:172.21.15.157:6789/0 conn(0x7fd52c15a850 legacy=0x7fd52c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd530e00700 1 -- 172.21.15.3:0/1341227518 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c0b58b0 con 0x7fd52c15ac30 2024-09-13T16:54:18.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd52be00700 1 -- 172.21.15.3:0/1341227518 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550694490 0 0) 0x7fd52c0b61e0 con 0x7fd52c15a850 2024-09-13T16:54:18.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd530e00700 1 --2- 172.21.15.3:0/1341227518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c0ce1c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd518009910 tx=0x7fd518009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9096f4b50bc238 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd52be00700 1 -- 172.21.15.3:0/1341227518 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518011830 con 0x7fd52c15ac30 2024-09-13T16:54:18.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd52be00700 1 -- 172.21.15.3:0/1341227518 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518011990 con 0x7fd52c15ac30 2024-09-13T16:54:18.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.362+0000 7fd52be00700 1 -- 172.21.15.3:0/1341227518 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518011d40 con 0x7fd52c15ac30 2024-09-13T16:54:18.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.366+0000 7fd532c00700 1 -- 172.21.15.3:0/1341227518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 msgr2=0x7fd52c0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.366+0000 7fd532c00700 1 --2- 172.21.15.3:0/1341227518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c0ce1c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd518009910 tx=0x7fd518009c20 comp rx=0 tx=0).stop 2024-09-13T16:54:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.366+0000 7fd532c00700 1 -- 172.21.15.3:0/1341227518 shutdown_connections 2024-09-13T16:54:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.366+0000 7fd532c00700 1 --2- 172.21.15.3:0/1341227518 >> v2:172.21.15.157:3300/0 conn(0x7fd52c0ce700 0x7fd52c06d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.366+0000 7fd532c00700 1 --2- 172.21.15.3:0/1341227518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c0ce1c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.366+0000 7fd532c00700 1 -- 172.21.15.3:0/1341227518 >> 172.21.15.3:0/1341227518 conn(0x7fd52c0b2a60 msgr2=0x7fd52c0b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.366+0000 7fd532c00700 1 -- 172.21.15.3:0/1341227518 shutdown_connections 2024-09-13T16:54:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.366+0000 7fd532c00700 1 -- 172.21.15.3:0/1341227518 wait complete. 2024-09-13T16:54:18.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532c00700 1 Processor -- start 2024-09-13T16:54:18.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532c00700 1 -- start start 2024-09-13T16:54:18.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c0ce700 0x7fd52c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c15a850 0x7fd52c150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532c00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c150a20 con 0x7fd52c15a850 2024-09-13T16:54:18.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c151990 con 0x7fd52c0ce700 2024-09-13T16:54:18.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532c00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c151c70 con 0x7fd52c15ac30 2024-09-13T16:54:18.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c15a850 0x7fd52c150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c15a850 0x7fd52c150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49646/0 (socket says 172.21.15.3:49646) 2024-09-13T16:54:18.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 -- 172.21.15.3:0/2356236494 learned_addr learned my addr 172.21.15.3:0/2356236494 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:18.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd532200700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd531800700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c0ce700 0x7fd52c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 -- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 msgr2=0x7fd52c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 -- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c0ce700 msgr2=0x7fd52c152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c0ce700 0x7fd52c152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 -- 172.21.15.3:0/2356236494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518009570 con 0x7fd52c15a850 2024-09-13T16:54:18.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.370+0000 7fd530e00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c15a850 0x7fd52c150120 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd518013000 tx=0x7fd518004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.374+0000 7fd529600700 1 -- 172.21.15.3:0/2356236494 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800b970 con 0x7fd52c15a850 2024-09-13T16:54:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.374+0000 7fd529600700 1 -- 172.21.15.3:0/2356236494 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51801e070 con 0x7fd52c15a850 2024-09-13T16:54:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.374+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c151f50 con 0x7fd52c15a850 2024-09-13T16:54:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.374+0000 7fd529600700 1 -- 172.21.15.3:0/2356236494 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800bd10 con 0x7fd52c15a850 2024-09-13T16:54:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.374+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c152510 con 0x7fd52c15a850 2024-09-13T16:54:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.374+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c06d0f0 con 0x7fd52c15a850 2024-09-13T16:54:18.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.378+0000 7fd529600700 1 -- 172.21.15.3:0/2356236494 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd518025480 con 0x7fd52c15a850 2024-09-13T16:54:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.378+0000 7fd529600700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd50c066730 0x7fd50c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.378+0000 7fd529600700 1 -- 172.21.15.3:0/2356236494 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd518020070 con 0x7fd52c15a850 2024-09-13T16:54:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.378+0000 7fd531800700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd50c066730 0x7fd50c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.378+0000 7fd531800700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd50c066730 0x7fd50c068bf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd520007fd0 tx=0x7fd520007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.386+0000 7fd529600700 1 -- 172.21.15.3:0/2356236494 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd5180529c0 con 0x7fd52c15a850 2024-09-13T16:54:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.570+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd52c0d05a0 con 0x7fd50c066730 2024-09-13T16:54:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.570+0000 7fd529600700 1 -- 172.21.15.3:0/2356236494 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd520013420 con 0x7fd50c066730 2024-09-13T16:54:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.574+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd50c066730 msgr2=0x7fd50c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.574+0000 7fd532c00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd50c066730 0x7fd50c068bf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd520007fd0 tx=0x7fd520007c20 comp rx=0 tx=0).stop 2024-09-13T16:54:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.574+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c15a850 msgr2=0x7fd52c150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.574+0000 7fd532c00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c15a850 0x7fd52c150120 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd518013000 tx=0x7fd518004770 comp rx=0 tx=0).stop 2024-09-13T16:54:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.578+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 shutdown_connections 2024-09-13T16:54:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.578+0000 7fd532c00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd50c066730 0x7fd50c068bf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.578+0000 7fd532c00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c15ac30 0x7fd52c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.578+0000 7fd532c00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c15a850 0x7fd52c150120 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.578+0000 7fd532c00700 1 --2- 172.21.15.3:0/2356236494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c0ce700 0x7fd52c152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.578+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 >> 172.21.15.3:0/2356236494 conn(0x7fd52c0b2a60 msgr2=0x7fd52c0b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.578+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 shutdown_connections 2024-09-13T16:54:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.578+0000 7fd532c00700 1 -- 172.21.15.3:0/2356236494 wait complete. 2024-09-13T16:54:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T16:54:18.603 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T16:54:18.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T16:54:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.794+0000 7f294de00700 1 Processor -- start 2024-09-13T16:54:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.794+0000 7f294de00700 1 -- start start 2024-09-13T16:54:18.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294de00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2948163de0 0x7f294815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294de00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29481628f0 con 0x7f29480b49e0 2024-09-13T16:54:18.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294de00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294806c780 con 0x7f29480b4dc0 2024-09-13T16:54:18.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294de00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294806ca60 con 0x7f2948163de0 2024-09-13T16:54:18.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294ca00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f29480b49e0 0x7f294815b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56628/0 (socket says 172.21.15.3:56628) 2024-09-13T16:54:18.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294ca00700 1 -- 172.21.15.3:0/4164989836 learned_addr learned my addr 172.21.15.3:0/4164989836 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:18.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294d400700 1 --2- 172.21.15.3:0/4164989836 >> v2:172.21.15.157:3300/0 conn(0x7f2948163de0 0x7f294815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f2947400700 1 -- 172.21.15.3:0/4164989836 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318646615 0 0) 0x7f29481628f0 con 0x7f29480b49e0 2024-09-13T16:54:18.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f2947400700 1 -- 172.21.15.3:0/4164989836 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29300035d0 con 0x7f29480b49e0 2024-09-13T16:54:18.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294d400700 1 -- 172.21.15.3:0/4164989836 >> v1:172.21.15.157:6789/0 conn(0x7f29480b4dc0 legacy=0x7f294806ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294d400700 1 -- 172.21.15.3:0/4164989836 >> v1:172.21.15.3:6789/0 conn(0x7f29480b49e0 legacy=0x7f294815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294d400700 1 -- 172.21.15.3:0/4164989836 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294806cd40 con 0x7f2948163de0 2024-09-13T16:54:18.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.798+0000 7f294d400700 1 --2- 172.21.15.3:0/4164989836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2948163de0 0x7f294815b7d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2940009d90 tx=0x7f294000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba20d6dd9b748249 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f2947400700 1 -- 172.21.15.3:0/4164989836 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000e070 con 0x7f2948163de0 2024-09-13T16:54:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f2947400700 1 -- 172.21.15.3:0/4164989836 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294000bb20 con 0x7f2948163de0 2024-09-13T16:54:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f2947400700 1 -- 172.21.15.3:0/4164989836 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940012430 con 0x7f2948163de0 2024-09-13T16:54:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 -- 172.21.15.3:0/4164989836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2948163de0 msgr2=0x7f294815b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 --2- 172.21.15.3:0/4164989836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2948163de0 0x7f294815b7d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2940009d90 tx=0x7f294000b330 comp rx=0 tx=0).stop 2024-09-13T16:54:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 -- 172.21.15.3:0/4164989836 shutdown_connections 2024-09-13T16:54:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 --2- 172.21.15.3:0/4164989836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2948163de0 0x7f294815b7d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 -- 172.21.15.3:0/4164989836 >> 172.21.15.3:0/4164989836 conn(0x7f29480b6210 msgr2=0x7f29480b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 -- 172.21.15.3:0/4164989836 shutdown_connections 2024-09-13T16:54:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 -- 172.21.15.3:0/4164989836 wait complete. 2024-09-13T16:54:18.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 Processor -- start 2024-09-13T16:54:18.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.802+0000 7f294de00700 1 -- start start 2024-09-13T16:54:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294de00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29480b49e0 0x7f2948148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29480b4dc0 0x7f2948154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2948163de0 0x7f2948155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294de00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2948154b30 con 0x7f29480b4dc0 2024-09-13T16:54:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294de00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294815c080 con 0x7f2948163de0 2024-09-13T16:54:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294de00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294815c360 con 0x7f29480b49e0 2024-09-13T16:54:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29480b4dc0 0x7f2948154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29480b4dc0 0x7f2948154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49656/0 (socket says 172.21.15.3:49656) 2024-09-13T16:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 -- 172.21.15.3:0/1461537321 learned_addr learned my addr 172.21.15.3:0/1461537321 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294d400700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2948163de0 0x7f2948155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294ca00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29480b49e0 0x7f2948148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 -- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29480b49e0 msgr2=0x7f2948148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29480b49e0 0x7f2948148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 -- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2948163de0 msgr2=0x7f2948155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2948163de0 0x7f2948155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 -- 172.21.15.3:0/1461537321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2940009a70 con 0x7f29480b4dc0 2024-09-13T16:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2947e00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29480b4dc0 0x7f2948154250 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f293400c500 tx=0x7f293400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340052d0 con 0x7f29480b4dc0 2024-09-13T16:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2934005430 con 0x7f29480b4dc0 2024-09-13T16:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340055a0 con 0x7f29480b4dc0 2024-09-13T16:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294815c640 con 0x7f29480b4dc0 2024-09-13T16:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.806+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2948154790 con 0x7f29480b4dc0 2024-09-13T16:54:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.810+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f294815bee0 con 0x7f29480b4dc0 2024-09-13T16:54:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.810+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29340038e0 con 0x7f29480b4dc0 2024-09-13T16:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.810+0000 7f2944c00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2928066780 0x7f2928068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.810+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f29340833e0 con 0x7f29480b4dc0 2024-09-13T16:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.814+0000 7f2944c00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7f292806cbc0 0x7f292806f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.814+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 --> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f292806f6d0 con 0x7f292806cbc0 2024-09-13T16:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.814+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29340859c0 con 0x7f29480b4dc0 2024-09-13T16:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.814+0000 7f294ca00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2928066780 0x7f2928068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.814+0000 7f294d400700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7f292806cbc0 0x7f292806f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:18.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.814+0000 7f294ca00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2928066780 0x7f2928068c40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f293c007fd0 tx=0x7f293c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.814+0000 7f294d400700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7f292806cbc0 0x7f292806f000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.814+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 <== osd.4 v2:172.21.15.157:6808/960819469 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f292806f6d0 con 0x7f292806cbc0 2024-09-13T16:54:18.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.834+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 --> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f294815c860 con 0x7f292806cbc0 2024-09-13T16:54:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.834+0000 7f2944c00700 1 -- 172.21.15.3:0/1461537321 <== osd.4 v2:172.21.15.157:6808/960819469 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f294815c860 con 0x7f292806cbc0 2024-09-13T16:54:18.840 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:54:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T16:54:18.840 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:54:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7f292806cbc0 msgr2=0x7f292806f000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6808/960819469,v1:172.21.15.157:6809/960819469] conn(0x7f292806cbc0 0x7f292806f000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2928066780 msgr2=0x7f2928068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f2928066780 0x7f2928068c40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f293c007fd0 tx=0x7f293c007de0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29480b4dc0 msgr2=0x7f2948154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 --2- 172.21.15.3:0/1461537321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29480b4dc0 0x7f2948154250 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f293400c500 tx=0x7f293400c810 comp rx=0 tx=0).stop 2024-09-13T16:54:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294d400700 1 -- 172.21.15.3:0/1461537321 reap_dead start 2024-09-13T16:54:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 shutdown_connections 2024-09-13T16:54:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 >> 172.21.15.3:0/1461537321 conn(0x7f29480b6210 msgr2=0x7f2948157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 shutdown_connections 2024-09-13T16:54:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:18.838+0000 7f294de00700 1 -- 172.21.15.3:0/1461537321 wait complete. 2024-09-13T16:54:18.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T16:54:19.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.038+0000 7fd293200700 1 Processor -- start 2024-09-13T16:54:19.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.038+0000 7fd293200700 1 -- start start 2024-09-13T16:54:19.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd293200700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd294144790 0x7fd29406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd293200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd29406cb20 0x7fd29415b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd293200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940b4100 con 0x7fd2941443b0 2024-09-13T16:54:19.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd293200700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29406cfb0 con 0x7fd294144790 2024-09-13T16:54:19.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd293200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940ce8e0 con 0x7fd29406cb20 2024-09-13T16:54:19.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd291400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd294144790 0x7fd29406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd291400700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd294144790 0x7fd29406c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49662/0 (socket says 172.21.15.3:49662) 2024-09-13T16:54:19.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd291400700 1 -- 172.21.15.3:0/1480450341 learned_addr learned my addr 172.21.15.3:0/1480450341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:19.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd292800700 1 --2- 172.21.15.3:0/1480450341 >> v2:172.21.15.157:3300/0 conn(0x7fd29406cb20 0x7fd29415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd291400700 1 -- 172.21.15.3:0/1480450341 >> v2:172.21.15.157:3300/0 conn(0x7fd29406cb20 msgr2=0x7fd29415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:19.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd291400700 1 --2- 172.21.15.3:0/1480450341 >> v2:172.21.15.157:3300/0 conn(0x7fd29406cb20 0x7fd29415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd291400700 1 -- 172.21.15.3:0/1480450341 >> v1:172.21.15.157:6789/0 conn(0x7fd2941443b0 legacy=0x7fd2940b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd291400700 1 -- 172.21.15.3:0/1480450341 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940ccd30 con 0x7fd294144790 2024-09-13T16:54:19.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd291400700 1 --2- 172.21.15.3:0/1480450341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd294144790 0x7fd29406c5e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd2940b3280 tx=0x7fd280009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95ef2e6abc23db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd290a00700 1 -- 172.21.15.3:0/1480450341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28000b8a0 con 0x7fd294144790 2024-09-13T16:54:19.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd290a00700 1 -- 172.21.15.3:0/1480450341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28000ba00 con 0x7fd294144790 2024-09-13T16:54:19.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.042+0000 7fd290a00700 1 -- 172.21.15.3:0/1480450341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28000bd90 con 0x7fd294144790 2024-09-13T16:54:19.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 -- 172.21.15.3:0/1480450341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd294144790 msgr2=0x7fd29406c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 --2- 172.21.15.3:0/1480450341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd294144790 0x7fd29406c5e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd2940b3280 tx=0x7fd280009bf0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 -- 172.21.15.3:0/1480450341 shutdown_connections 2024-09-13T16:54:19.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 --2- 172.21.15.3:0/1480450341 >> v2:172.21.15.157:3300/0 conn(0x7fd29406cb20 0x7fd29415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 --2- 172.21.15.3:0/1480450341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd294144790 0x7fd29406c5e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 -- 172.21.15.3:0/1480450341 >> 172.21.15.3:0/1480450341 conn(0x7fd2940b0cf0 msgr2=0x7fd2940b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:19.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 -- 172.21.15.3:0/1480450341 shutdown_connections 2024-09-13T16:54:19.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 -- 172.21.15.3:0/1480450341 wait complete. 2024-09-13T16:54:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 Processor -- start 2024-09-13T16:54:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.046+0000 7fd293200700 1 -- start start 2024-09-13T16:54:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd293200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29406cb20 0x7fd294156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd293200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2941443b0 0x7fd2940c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd293200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29413cf30 0x7fd29413e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd293200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294156ed0 con 0x7fd29406cb20 2024-09-13T16:54:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd293200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940c4420 con 0x7fd29413cf30 2024-09-13T16:54:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd293200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940c4700 con 0x7fd2941443b0 2024-09-13T16:54:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29406cb20 0x7fd294156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29406cb20 0x7fd294156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49670/0 (socket says 172.21.15.3:49670) 2024-09-13T16:54:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 -- 172.21.15.3:0/2993251124 learned_addr learned my addr 172.21.15.3:0/2993251124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291400700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2941443b0 0x7fd2940c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd292800700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29413cf30 0x7fd29413e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 -- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2941443b0 msgr2=0x7fd2940c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2941443b0 0x7fd2940c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 -- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29413cf30 msgr2=0x7fd29413e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29413cf30 0x7fd29413e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 -- 172.21.15.3:0/2993251124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280009570 con 0x7fd29406cb20 2024-09-13T16:54:19.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd291e00700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29406cb20 0x7fd294156990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd28400c500 tx=0x7fd28400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd28a000700 1 -- 172.21.15.3:0/2993251124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2840052d0 con 0x7fd29406cb20 2024-09-13T16:54:19.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd28a000700 1 -- 172.21.15.3:0/2993251124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd284005430 con 0x7fd29406cb20 2024-09-13T16:54:19.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2940c49e0 con 0x7fd29406cb20 2024-09-13T16:54:19.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd28a000700 1 -- 172.21.15.3:0/2993251124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2840055a0 con 0x7fd29406cb20 2024-09-13T16:54:19.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.050+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2940c4fa0 con 0x7fd29406cb20 2024-09-13T16:54:19.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.054+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2940cca20 con 0x7fd29406cb20 2024-09-13T16:54:19.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.054+0000 7fd28a000700 1 -- 172.21.15.3:0/2993251124 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd2840036a0 con 0x7fd29406cb20 2024-09-13T16:54:19.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.054+0000 7fd28a000700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd274066730 0x7fd274068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.054+0000 7fd28a000700 1 -- 172.21.15.3:0/2993251124 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd284083220 con 0x7fd29406cb20 2024-09-13T16:54:19.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.058+0000 7fd291400700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd274066730 0x7fd274068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.058+0000 7fd291400700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd274066730 0x7fd274068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd280008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.062+0000 7fd28a000700 1 -- 172.21.15.3:0/2993251124 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd284050690 con 0x7fd29406cb20 2024-09-13T16:54:19.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.242+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd29404e770 con 0x7fd274066730 2024-09-13T16:54:19.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.246+0000 7fd28a000700 1 -- 172.21.15.3:0/2993251124 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd29404e770 con 0x7fd274066730 2024-09-13T16:54:19.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.246+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd274066730 msgr2=0x7fd274068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.246+0000 7fd293200700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd274066730 0x7fd274068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd280008030 comp rx=0 tx=0).stop 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.246+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29406cb20 msgr2=0x7fd294156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.246+0000 7fd293200700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29406cb20 0x7fd294156990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd28400c500 tx=0x7fd28400c810 comp rx=0 tx=0).stop 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.250+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 shutdown_connections 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.250+0000 7fd293200700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fd274066730 0x7fd274068bf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.250+0000 7fd293200700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29413cf30 0x7fd29413e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.250+0000 7fd293200700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2941443b0 0x7fd2940c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.250+0000 7fd293200700 1 --2- 172.21.15.3:0/2993251124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29406cb20 0x7fd294156990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.250+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 >> 172.21.15.3:0/2993251124 conn(0x7fd2940b0cf0 msgr2=0x7fd2940addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:19.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.250+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 shutdown_connections 2024-09-13T16:54:19.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.250+0000 7fd293200700 1 -- 172.21.15.3:0/2993251124 wait complete. 2024-09-13T16:54:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T16:54:19.274 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T16:54:19.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T16:54:19.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.450+0000 7f3f60e00700 1 Processor -- start 2024-09-13T16:54:19.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.454+0000 7f3f60e00700 1 -- start start 2024-09-13T16:54:19.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.454+0000 7f3f60e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b4e10 0x7f3f5c0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.454+0000 7f3f60e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f5c0b17f0 0x7f3f5c0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.454+0000 7f3f60e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c0ce6a0 con 0x7f3f5c146580 2024-09-13T16:54:19.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.454+0000 7f3f60e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0ce090 con 0x7f3f5c0b17f0 2024-09-13T16:54:19.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.454+0000 7f3f60e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0b2010 con 0x7f3f5c0b4e10 2024-09-13T16:54:19.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5b400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3f5c146580 0x7f3f5c0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56632/0 (socket says 172.21.15.3:56632) 2024-09-13T16:54:19.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5be00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f5c0b17f0 0x7f3f5c0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5b400700 1 -- 172.21.15.3:0/3105866435 learned_addr learned my addr 172.21.15.3:0/3105866435 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5aa00700 1 --2- 172.21.15.3:0/3105866435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b4e10 0x7f3f5c0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5be00700 1 -- 172.21.15.3:0/3105866435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b4e10 msgr2=0x7f3f5c0b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5be00700 1 --2- 172.21.15.3:0/3105866435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b4e10 0x7f3f5c0b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5be00700 1 -- 172.21.15.3:0/3105866435 >> v1:172.21.15.3:6789/0 conn(0x7f3f5c146580 legacy=0x7f3f5c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5be00700 1 -- 172.21.15.3:0/3105866435 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0b22f0 con 0x7f3f5c0b17f0 2024-09-13T16:54:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5a000700 1 -- 172.21.15.3:0/3105866435 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181563830 0 0) 0x7f3f5c0ce6a0 con 0x7f3f5c146580 2024-09-13T16:54:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5be00700 1 --2- 172.21.15.3:0/3105866435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c0b17f0 0x7f3f5c0ceb30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009d90 tx=0x7f3f5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e991752b7304e2cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5a000700 1 -- 172.21.15.3:0/3105866435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400e070 con 0x7f3f5c0b17f0 2024-09-13T16:54:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5a000700 1 -- 172.21.15.3:0/3105866435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5400bb20 con 0x7f3f5c0b17f0 2024-09-13T16:54:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f5a000700 1 -- 172.21.15.3:0/3105866435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54012430 con 0x7f3f5c0b17f0 2024-09-13T16:54:19.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f60e00700 1 -- 172.21.15.3:0/3105866435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c0b17f0 msgr2=0x7f3f5c0ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f60e00700 1 --2- 172.21.15.3:0/3105866435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c0b17f0 0x7f3f5c0ceb30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009d90 tx=0x7f3f5400b330 comp rx=0 tx=0).stop 2024-09-13T16:54:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f60e00700 1 -- 172.21.15.3:0/3105866435 shutdown_connections 2024-09-13T16:54:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f60e00700 1 --2- 172.21.15.3:0/3105866435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c0b17f0 0x7f3f5c0ceb30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f60e00700 1 --2- 172.21.15.3:0/3105866435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b4e10 0x7f3f5c0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.458+0000 7f3f60e00700 1 -- 172.21.15.3:0/3105866435 >> 172.21.15.3:0/3105866435 conn(0x7f3f5c0b6210 msgr2=0x7f3f5c0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.462+0000 7f3f60e00700 1 -- 172.21.15.3:0/3105866435 shutdown_connections 2024-09-13T16:54:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.462+0000 7f3f60e00700 1 -- 172.21.15.3:0/3105866435 wait complete. 2024-09-13T16:54:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.462+0000 7f3f60e00700 1 Processor -- start 2024-09-13T16:54:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.462+0000 7f3f60e00700 1 -- start start 2024-09-13T16:54:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f60e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b17f0 0x7f3f5c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f60e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5c0b4e10 0x7f3f5c0a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f60e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c146580 0x7f3f5c0a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f60e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0a84a0 con 0x7f3f5c146580 2024-09-13T16:54:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f60e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0a86f0 con 0x7f3f5c0b4e10 2024-09-13T16:54:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f60e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0a89d0 con 0x7f3f5c0b17f0 2024-09-13T16:54:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c146580 0x7f3f5c0a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c146580 0x7f3f5c0a9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49680/0 (socket says 172.21.15.3:49680) 2024-09-13T16:54:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 -- 172.21.15.3:0/623828831 learned_addr learned my addr 172.21.15.3:0/623828831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5aa00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5c0b4e10 0x7f3f5c0a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5b400700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b17f0 0x7f3f5c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 -- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b17f0 msgr2=0x7f3f5c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f5c0b17f0 0x7f3f5c0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 -- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5c0b4e10 msgr2=0x7f3f5c0a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5c0b4e10 0x7f3f5c0a9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 -- 172.21.15.3:0/623828831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f54009a70 con 0x7f3f5c146580 2024-09-13T16:54:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f5be00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c146580 0x7f3f5c0a9cd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3f54014040 tx=0x7f3f54004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54012430 con 0x7f3f5c146580 2024-09-13T16:54:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f54020070 con 0x7f3f5c146580 2024-09-13T16:54:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54004280 con 0x7f3f5c146580 2024-09-13T16:54:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c0a8cb0 con 0x7f3f5c146580 2024-09-13T16:54:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.466+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c0a9270 con 0x7f3f5c146580 2024-09-13T16:54:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.470+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f5c0b1c80 con 0x7f3f5c146580 2024-09-13T16:54:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.470+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f540043e0 con 0x7f3f5c146580 2024-09-13T16:54:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.470+0000 7f3f53400700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f3c066760 0x7f3f3c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f3f54022070 con 0x7f3f5c146580 2024-09-13T16:54:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f53400700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7f3f3c06cba0 0x7f3f3c06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 --> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f3c06f6b0 con 0x7f3f3c06cba0 2024-09-13T16:54:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f5408a2a0 con 0x7f3f5c146580 2024-09-13T16:54:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f5b400700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f3c066760 0x7f3f3c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f5aa00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7f3f3c06cba0 0x7f3f3c06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f5aa00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7f3f3c06cba0 0x7f3f3c06efe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f5b400700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f3c066760 0x7f3f3c068c20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c000f60 tx=0x7f3f4c007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.474+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 <== osd.5 v2:172.21.15.157:6800/2560288896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3f3c06f6b0 con 0x7f3f3c06cba0 2024-09-13T16:54:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 --> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3f5c0b12b0 con 0x7f3f3c06cba0 2024-09-13T16:54:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f53400700 1 -- 172.21.15.3:0/623828831 <== osd.5 v2:172.21.15.157:6800/2560288896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3f5c0b12b0 con 0x7f3f3c06cba0 2024-09-13T16:54:19.501 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:54:19.501 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T16:54:19.501 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:54:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7f3f3c06cba0 msgr2=0x7f3f3c06efe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6800/2560288896,v1:172.21.15.157:6801/2560288896] conn(0x7f3f3c06cba0 0x7f3f3c06efe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f3c066760 msgr2=0x7f3f3c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f3f3c066760 0x7f3f3c068c20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c000f60 tx=0x7f3f4c007e90 comp rx=0 tx=0).stop 2024-09-13T16:54:19.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c146580 msgr2=0x7f3f5c0a9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 --2- 172.21.15.3:0/623828831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5c146580 0x7f3f5c0a9cd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3f54014040 tx=0x7f3f54004af0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f5be00700 1 -- 172.21.15.3:0/623828831 reap_dead start 2024-09-13T16:54:19.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 shutdown_connections 2024-09-13T16:54:19.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 >> 172.21.15.3:0/623828831 conn(0x7f3f5c0b6210 msgr2=0x7f3f5c06d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:19.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 shutdown_connections 2024-09-13T16:54:19.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.498+0000 7f3f60e00700 1 -- 172.21.15.3:0/623828831 wait complete. 2024-09-13T16:54:19.526 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T16:54:19.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.714+0000 7fa3fbe00700 1 Processor -- start 2024-09-13T16:54:19.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.714+0000 7fa3fbe00700 1 -- start start 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fbe00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3fc06d6b0 0x7fa3fc0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fbe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3fc0cd910 0x7fa3fc0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fbe00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc0cdd60 con 0x7fa3fc0cc720 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fbe00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc06db20 con 0x7fa3fc0cd910 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fbe00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc0b4ec0 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3faa00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa3fc0cc720 0x7fa3fc15af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56640/0 (socket says 172.21.15.3:56640) 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3faa00700 1 -- 172.21.15.3:0/4216287067 learned_addr learned my addr 172.21.15.3:0/4216287067 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fb400700 1 --2- 172.21.15.3:0/4216287067 >> v2:172.21.15.3:3300/0 conn(0x7fa3fc0cd910 0x7fa3fc0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fa000700 1 --2- 172.21.15.3:0/4216287067 >> v2:172.21.15.157:3300/0 conn(0x7fa3fc06d6b0 0x7fa3fc0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3f9600700 1 -- 172.21.15.3:0/4216287067 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824830899 0 0) 0x7fa3ec001110 con 0x7fa3fc0cc720 2024-09-13T16:54:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fb400700 1 -- 172.21.15.3:0/4216287067 >> v2:172.21.15.157:3300/0 conn(0x7fa3fc06d6b0 msgr2=0x7fa3fc0cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fb400700 1 --2- 172.21.15.3:0/4216287067 >> v2:172.21.15.157:3300/0 conn(0x7fa3fc06d6b0 0x7fa3fc0cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fb400700 1 -- 172.21.15.3:0/4216287067 >> v1:172.21.15.3:6789/0 conn(0x7fa3fc0cc720 legacy=0x7fa3fc15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fb400700 1 -- 172.21.15.3:0/4216287067 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc15ba60 con 0x7fa3fc0cd910 2024-09-13T16:54:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fb400700 1 --2- 172.21.15.3:0/4216287067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc0cd910 0x7fa3fc0b5350 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc068480 tx=0x7fa3f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1ca14564cfffd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3f9600700 1 -- 172.21.15.3:0/4216287067 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400e070 con 0x7fa3fc0cd910 2024-09-13T16:54:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3f9600700 1 -- 172.21.15.3:0/4216287067 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f400bb20 con 0x7fa3fc0cd910 2024-09-13T16:54:19.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3f9600700 1 -- 172.21.15.3:0/4216287067 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4012430 con 0x7fa3fc0cd910 2024-09-13T16:54:19.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fbe00700 1 -- 172.21.15.3:0/4216287067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc0cd910 msgr2=0x7fa3fc0b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.718+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/4216287067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc0cd910 0x7fa3fc0b5350 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc068480 tx=0x7fa3f400b330 comp rx=0 tx=0).stop 2024-09-13T16:54:19.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.722+0000 7fa3fbe00700 1 -- 172.21.15.3:0/4216287067 shutdown_connections 2024-09-13T16:54:19.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.722+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/4216287067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc0cd910 0x7fa3fc0b5350 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.722+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/4216287067 >> v2:172.21.15.157:3300/0 conn(0x7fa3fc06d6b0 0x7fa3fc0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.722+0000 7fa3fbe00700 1 -- 172.21.15.3:0/4216287067 >> 172.21.15.3:0/4216287067 conn(0x7fa3fc0b2a60 msgr2=0x7fa3fc0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:19.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.722+0000 7fa3fbe00700 1 -- 172.21.15.3:0/4216287067 shutdown_connections 2024-09-13T16:54:19.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.722+0000 7fa3fbe00700 1 -- 172.21.15.3:0/4216287067 wait complete. 2024-09-13T16:54:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.722+0000 7fa3fbe00700 1 Processor -- start 2024-09-13T16:54:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.722+0000 7fa3fbe00700 1 -- start start 2024-09-13T16:54:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fbe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc06d6b0 0x7fa3fc148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fbe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3fc0cc720 0x7fa3fc14b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3fc0cd910 0x7fa3fc149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fbe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc15c080 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fbe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc15e750 con 0x7fa3fc0cd910 2024-09-13T16:54:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fbe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc15ea30 con 0x7fa3fc0cc720 2024-09-13T16:54:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc06d6b0 0x7fa3fc148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc06d6b0 0x7fa3fc148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49696/0 (socket says 172.21.15.3:49696) 2024-09-13T16:54:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 -- 172.21.15.3:0/3088187561 learned_addr learned my addr 172.21.15.3:0/3088187561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fa000700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3fc0cc720 0x7fa3fc14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fb400700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3fc0cd910 0x7fa3fc149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 -- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3fc0cc720 msgr2=0x7fa3fc14b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3fc0cc720 0x7fa3fc14b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 -- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3fc0cd910 msgr2=0x7fa3fc149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3fc0cd910 0x7fa3fc149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 -- 172.21.15.3:0/3088187561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f4009a70 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fa000700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3fc0cc720 0x7fa3fc14b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3faa00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc06d6b0 0x7fa3fc148ab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c750 tx=0x7fa3ec00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3f2a00700 1 -- 172.21.15.3:0/3088187561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec004ca0 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3f2a00700 1 -- 172.21.15.3:0/3088187561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec004e20 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc15ed10 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.726+0000 7fa3f2a00700 1 -- 172.21.15.3:0/3088187561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec007310 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.730+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc148ff0 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.730+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc15c560 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.730+0000 7fa3f2a00700 1 -- 172.21.15.3:0/3088187561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3ec007470 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.734+0000 7fa3f2a00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3dc066730 0x7fa3dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:19.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.734+0000 7fa3f2a00700 1 -- 172.21.15.3:0/3088187561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fa3ec083d10 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.734+0000 7fa3fa000700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3dc066730 0x7fa3dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:19.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.734+0000 7fa3fa000700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3dc066730 0x7fa3dc068bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0cae30 tx=0x7fa3e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.738+0000 7fa3f2a00700 1 -- 172.21.15.3:0/3088187561 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa3fc15c560 con 0x7fa3fc06d6b0 2024-09-13T16:54:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.922+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3fc0b4b50 con 0x7fa3dc066730 2024-09-13T16:54:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.926+0000 7fa3f2a00700 1 -- 172.21.15.3:0/3088187561 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa3e8004030 con 0x7fa3dc066730 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.926+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3dc066730 msgr2=0x7fa3dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.926+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3dc066730 0x7fa3dc068bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0cae30 tx=0x7fa3e8008040 comp rx=0 tx=0).stop 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.926+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc06d6b0 msgr2=0x7fa3fc148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.926+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc06d6b0 0x7fa3fc148ab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c750 tx=0x7fa3ec00ca60 comp rx=0 tx=0).stop 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.930+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 shutdown_connections 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.930+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fa3dc066730 0x7fa3dc068bf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.930+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3fc0cd910 0x7fa3fc149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.930+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3fc0cc720 0x7fa3fc14b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.930+0000 7fa3fbe00700 1 --2- 172.21.15.3:0/3088187561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3fc06d6b0 0x7fa3fc148ab0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:19.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.930+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 >> 172.21.15.3:0/3088187561 conn(0x7fa3fc0b2a60 msgr2=0x7fa3fc06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:19.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.930+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 shutdown_connections 2024-09-13T16:54:19.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:19.930+0000 7fa3fbe00700 1 -- 172.21.15.3:0/3088187561 wait complete. 2024-09-13T16:54:19.934 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T16:54:19.953 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T16:54:19.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T16:54:20.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.138+0000 7f8513e00700 1 Processor -- start 2024-09-13T16:54:20.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.138+0000 7f8513e00700 1 -- start start 2024-09-13T16:54:20.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.142+0000 7f8513e00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85140ce490 0x7f85140b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.142+0000 7f8513e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85140b4a10 con 0x7f85140ce000 2024-09-13T16:54:20.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.142+0000 7f8513e00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8514155360 con 0x7f851415e480 2024-09-13T16:54:20.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.142+0000 7f8513e00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85140ce9e0 con 0x7f85140ce490 2024-09-13T16:54:20.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.142+0000 7f8512000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f85140ce000 0x7f851415c050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56646/0 (socket says 172.21.15.3:56646) 2024-09-13T16:54:20.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.142+0000 7f8512000700 1 -- 172.21.15.3:0/1920826007 learned_addr learned my addr 172.21.15.3:0/1920826007 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:20.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513400700 1 --2- 172.21.15.3:0/1920826007 >> v2:172.21.15.3:3300/0 conn(0x7f85140ce490 0x7f85140b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8511600700 1 -- 172.21.15.3:0/1920826007 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536716110 0 0) 0x7f85140b4a10 con 0x7f85140ce000 2024-09-13T16:54:20.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513400700 1 -- 172.21.15.3:0/1920826007 >> v1:172.21.15.157:6789/0 conn(0x7f851415e480 legacy=0x7f851415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513400700 1 -- 172.21.15.3:0/1920826007 >> v1:172.21.15.3:6789/0 conn(0x7f85140ce000 legacy=0x7f851415c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513400700 1 -- 172.21.15.3:0/1920826007 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140b4c80 con 0x7f85140ce490 2024-09-13T16:54:20.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513400700 1 --2- 172.21.15.3:0/1920826007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 0x7f85140b5760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f850c009d90 tx=0x7f850c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef181ea674df20c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8511600700 1 -- 172.21.15.3:0/1920826007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00e070 con 0x7f85140ce490 2024-09-13T16:54:20.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8511600700 1 -- 172.21.15.3:0/1920826007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c00bad0 con 0x7f85140ce490 2024-09-13T16:54:20.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8511600700 1 -- 172.21.15.3:0/1920826007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00be80 con 0x7f85140ce490 2024-09-13T16:54:20.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513e00700 1 -- 172.21.15.3:0/1920826007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 msgr2=0x7f85140b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513e00700 1 --2- 172.21.15.3:0/1920826007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 0x7f85140b5760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f850c009d90 tx=0x7f850c00b330 comp rx=0 tx=0).stop 2024-09-13T16:54:20.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513e00700 1 -- 172.21.15.3:0/1920826007 shutdown_connections 2024-09-13T16:54:20.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513e00700 1 --2- 172.21.15.3:0/1920826007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 0x7f85140b5760 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513e00700 1 -- 172.21.15.3:0/1920826007 >> 172.21.15.3:0/1920826007 conn(0x7f85140b2910 msgr2=0x7f85140b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:20.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513e00700 1 -- 172.21.15.3:0/1920826007 shutdown_connections 2024-09-13T16:54:20.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.146+0000 7f8513e00700 1 -- 172.21.15.3:0/1920826007 wait complete. 2024-09-13T16:54:20.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.150+0000 7f8513e00700 1 Processor -- start 2024-09-13T16:54:20.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.150+0000 7f8513e00700 1 -- start start 2024-09-13T16:54:20.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.150+0000 7f8513e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85140ce000 0x7f8514149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.150+0000 7f8513e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 0x7f851406c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.150+0000 7f8513e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f851415e480 0x7f8514146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.150+0000 7f8513e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514155550 con 0x7f85140ce490 2024-09-13T16:54:20.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.150+0000 7f8513e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140ca5c0 con 0x7f85140ce000 2024-09-13T16:54:20.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.150+0000 7f8513e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8514143d20 con 0x7f851415e480 2024-09-13T16:54:20.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 0x7f851406c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 0x7f851406c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49712/0 (socket says 172.21.15.3:49712) 2024-09-13T16:54:20.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 -- 172.21.15.3:0/941453422 learned_addr learned my addr 172.21.15.3:0/941453422 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:20.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8513400700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f851415e480 0x7f8514146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512a00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85140ce000 0x7f8514149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 -- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f851415e480 msgr2=0x7f8514146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f851415e480 0x7f8514146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 -- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85140ce000 msgr2=0x7f8514149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85140ce000 0x7f8514149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 -- 172.21.15.3:0/941453422 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009a70 con 0x7f85140ce490 2024-09-13T16:54:20.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512a00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85140ce000 0x7f8514149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:20.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8512000700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 0x7f851406c880 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f850000c580 tx=0x7f850000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500004ca0 con 0x7f85140ce490 2024-09-13T16:54:20.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8500004e20 con 0x7f85140ce490 2024-09-13T16:54:20.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8514155780 con 0x7f85140ce490 2024-09-13T16:54:20.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500007310 con 0x7f85140ce490 2024-09-13T16:54:20.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851406d0d0 con 0x7f85140ce490 2024-09-13T16:54:20.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.154+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85140b2390 con 0x7f85140ce490 2024-09-13T16:54:20.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.158+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8500007470 con 0x7f85140ce490 2024-09-13T16:54:20.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.158+0000 7f850aa00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f84f4066730 0x7f84f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.158+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f8500083d70 con 0x7f85140ce490 2024-09-13T16:54:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.162+0000 7f850aa00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f84f406cb70 0x7f84f406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.162+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 --> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84f406f680 con 0x7f84f406cb70 2024-09-13T16:54:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.162+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8500086350 con 0x7f85140ce490 2024-09-13T16:54:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.162+0000 7f8512a00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f84f4066730 0x7f84f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.162+0000 7f8513400700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f84f406cb70 0x7f84f406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.162+0000 7f8513400700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f84f406cb70 0x7f84f406efb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.162+0000 7f8512a00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f84f4066730 0x7f84f4068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f85040018e0 tx=0x7f8504009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.162+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 <== osd.6 v2:172.21.15.157:6824/1906982980 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f84f406f680 con 0x7f84f406cb70 2024-09-13T16:54:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.182+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 --> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f851415c510 con 0x7f84f406cb70 2024-09-13T16:54:20.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f850aa00700 1 -- 172.21.15.3:0/941453422 <== osd.6 v2:172.21.15.157:6824/1906982980 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f851415c510 con 0x7f84f406cb70 2024-09-13T16:54:20.188 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:54:20.188 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T16:54:20.188 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:54:20.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f84f406cb70 msgr2=0x7f84f406efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6824/1906982980,v1:172.21.15.157:6825/1906982980] conn(0x7f84f406cb70 0x7f84f406efb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f84f4066730 msgr2=0x7f84f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f84f4066730 0x7f84f4068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f85040018e0 tx=0x7f8504009040 comp rx=0 tx=0).stop 2024-09-13T16:54:20.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 msgr2=0x7f851406c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 --2- 172.21.15.3:0/941453422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85140ce490 0x7f851406c880 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f850000c580 tx=0x7f850000c890 comp rx=0 tx=0).stop 2024-09-13T16:54:20.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513400700 1 -- 172.21.15.3:0/941453422 reap_dead start 2024-09-13T16:54:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 shutdown_connections 2024-09-13T16:54:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 >> 172.21.15.3:0/941453422 conn(0x7f85140b2910 msgr2=0x7f85140b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 shutdown_connections 2024-09-13T16:54:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.186+0000 7f8513e00700 1 -- 172.21.15.3:0/941453422 wait complete. 2024-09-13T16:54:20.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T16:54:20.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.386+0000 7fece6e00700 1 Processor -- start 2024-09-13T16:54:20.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.386+0000 7fece6e00700 1 -- start start 2024-09-13T16:54:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.386+0000 7fece6e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fece006fb50 0x7fece00cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.386+0000 7fece6e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece006ff30 0x7fece00b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.386+0000 7fece6e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece00cf070 con 0x7fece00b4390 2024-09-13T16:54:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.386+0000 7fece6e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece00b4be0 con 0x7fece006fb50 2024-09-13T16:54:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.386+0000 7fece6e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece00cf330 con 0x7fece006ff30 2024-09-13T16:54:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece6400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fece00b4390 0x7fece015e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56658/0 (socket says 172.21.15.3:56658) 2024-09-13T16:54:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece6400700 1 -- 172.21.15.3:0/2897458641 learned_addr learned my addr 172.21.15.3:0/2897458641 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5000700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece006ff30 0x7fece00b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5a00700 1 --2- 172.21.15.3:0/2897458641 >> v2:172.21.15.157:3300/0 conn(0x7fece006fb50 0x7fece00cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159503513 0 0) 0x7fece00cf070 con 0x7fece00b4390 2024-09-13T16:54:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc80035d0 con 0x7fece00b4390 2024-09-13T16:54:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2485731272 0 0) 0x7fecc80035d0 con 0x7fece00b4390 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece006ff30 msgr2=0x7fece00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece006ff30 0x7fece00b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 >> v2:172.21.15.157:3300/0 conn(0x7fece006fb50 msgr2=0x7fece00cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 --2- 172.21.15.3:0/2897458641 >> v2:172.21.15.157:3300/0 conn(0x7fece006fb50 0x7fece00cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00cf610 con 0x7fece00b4390 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1031945381 0 0) 0x7fecd80019f0 con 0x7fece00b4390 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 >> v1:172.21.15.3:6789/0 conn(0x7fece00b4390 legacy=0x7fece015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fecc8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece006ff30 0x7fecc8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc8004db0 0x7fecc8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00b4be0 con 0x7fecc8003a20 2024-09-13T16:54:20.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00cf330 con 0x7fecc8004db0 2024-09-13T16:54:20.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc800ba60 con 0x7fece006ff30 2024-09-13T16:54:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5a00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fecc8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5000700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece006ff30 0x7fecc8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece6400700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc8004db0 0x7fecc8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5a00700 1 -- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece006ff30 msgr2=0x7fecc8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5a00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece006ff30 0x7fecc8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5a00700 1 -- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc8004db0 msgr2=0x7fecc8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5a00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc8004db0 0x7fecc8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece5a00700 1 -- 172.21.15.3:0/2897458641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc80035d0 con 0x7fecc8003a20 2024-09-13T16:54:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.390+0000 7fece6400700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc8004db0 0x7fecc8005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fece5a00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fecc8003e40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009b70 tx=0x7fecc80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7131993019bb943a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400e6c0 con 0x7fecc8003a20 2024-09-13T16:54:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd400e820 con 0x7fecc8003a20 2024-09-13T16:54:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fecdfe00700 1 -- 172.21.15.3:0/2897458641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400eb90 con 0x7fecc8003a20 2024-09-13T16:54:20.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fece6e00700 1 -- 172.21.15.3:0/2897458641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 msgr2=0x7fecc8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fece6e00700 1 --2- 172.21.15.3:0/2897458641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fecc8003e40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009b70 tx=0x7fecc80058a0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fece6400700 1 -- 172.21.15.3:0/2897458641 reap_dead start 2024-09-13T16:54:20.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fece6e00700 1 -- 172.21.15.3:0/2897458641 shutdown_connections 2024-09-13T16:54:20.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fece6e00700 1 -- 172.21.15.3:0/2897458641 >> 172.21.15.3:0/2897458641 conn(0x7fece00b2910 msgr2=0x7fece00b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:20.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fece6e00700 1 -- 172.21.15.3:0/2897458641 shutdown_connections 2024-09-13T16:54:20.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.394+0000 7fece6e00700 1 -- 172.21.15.3:0/2897458641 wait complete. 2024-09-13T16:54:20.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece6e00700 1 Processor -- start 2024-09-13T16:54:20.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece6e00700 1 -- start start 2024-09-13T16:54:20.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece6e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fece013e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece6e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece0149d30 0x7fece014a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece6e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece0150080 0x7fece0150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece6e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece014a6d0 con 0x7fecc8003a20 2024-09-13T16:54:20.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece6e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece0150a50 con 0x7fece0149d30 2024-09-13T16:54:20.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece6e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece014fa70 con 0x7fece0150080 2024-09-13T16:54:20.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece5a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fece013e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece5a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fece013e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49736/0 (socket says 172.21.15.3:49736) 2024-09-13T16:54:20.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.398+0000 7fece5a00700 1 -- 172.21.15.3:0/619294209 learned_addr learned my addr 172.21.15.3:0/619294209 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:20.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece6400700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece0150080 0x7fece0150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece5000700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece0149d30 0x7fece014a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece5a00700 1 -- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece0150080 msgr2=0x7fece0150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece5a00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece0150080 0x7fece0150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece5a00700 1 -- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece0149d30 msgr2=0x7fece014a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece5a00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece0149d30 0x7fece014a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece5a00700 1 -- 172.21.15.3:0/619294209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece013eaa0 con 0x7fecc8003a20 2024-09-13T16:54:20.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece5a00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fece013e560 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fecc8004780 tx=0x7fecd40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fecdd600700 1 -- 172.21.15.3:0/619294209 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400e6c0 con 0x7fecc8003a20 2024-09-13T16:54:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fecdd600700 1 -- 172.21.15.3:0/619294209 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4004030 con 0x7fecc8003a20 2024-09-13T16:54:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fecdd600700 1 -- 172.21.15.3:0/619294209 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40043e0 con 0x7fecc8003a20 2024-09-13T16:54:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece014dd00 con 0x7fecc8003a20 2024-09-13T16:54:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece014e2c0 con 0x7fecc8003a20 2024-09-13T16:54:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.402+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece00cfe10 con 0x7fecc8003a20 2024-09-13T16:54:20.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.406+0000 7fecdd600700 1 -- 172.21.15.3:0/619294209 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fecd40036c0 con 0x7fecc8003a20 2024-09-13T16:54:20.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.406+0000 7fecdd600700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fecc0066730 0x7fecc0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.406+0000 7fecdd600700 1 -- 172.21.15.3:0/619294209 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fecd4085980 con 0x7fecc8003a20 2024-09-13T16:54:20.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.406+0000 7fece5000700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fecc0066730 0x7fecc0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.406+0000 7fece5000700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fecc0066730 0x7fecc0068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feccc009bb0 tx=0x7feccc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.414+0000 7fecdd600700 1 -- 172.21.15.3:0/619294209 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fecd4052f10 con 0x7fecc8003a20 2024-09-13T16:54:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.602+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fece014a6d0 con 0x7fecc0066730 2024-09-13T16:54:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fecdd600700 1 -- 172.21.15.3:0/619294209 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fece014a6d0 con 0x7fecc0066730 2024-09-13T16:54:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fecc0066730 msgr2=0x7fecc0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fecc0066730 0x7fecc0068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feccc009bb0 tx=0x7feccc008040 comp rx=0 tx=0).stop 2024-09-13T16:54:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 msgr2=0x7fece013e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fece013e560 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fecc8004780 tx=0x7fecd40047e0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 shutdown_connections 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fecc0066730 0x7fecc0068bf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece0150080 0x7fece0150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece0149d30 0x7fece014a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 --2- 172.21.15.3:0/619294209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecc8003a20 0x7fece013e560 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 >> 172.21.15.3:0/619294209 conn(0x7fece00b2910 msgr2=0x7fece00b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 shutdown_connections 2024-09-13T16:54:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.606+0000 7fece6e00700 1 -- 172.21.15.3:0/619294209 wait complete. 2024-09-13T16:54:20.613 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T16:54:20.636 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T16:54:20.637 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T16:54:20.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.814+0000 7ff507600700 1 Processor -- start 2024-09-13T16:54:20.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.814+0000 7ff507600700 1 -- start start 2024-09-13T16:54:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff507600700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff50006ce80 0x7ff500098ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff507600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 0x7ff50006c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff507600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff5000ad520 0x7ff5001644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff507600700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff500070af0 con 0x7ff5000ad520 2024-09-13T16:54:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff507600700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50006ca50 con 0x7ff50006ce80 2024-09-13T16:54:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff507600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000ae210 con 0x7ff50006d260 2024-09-13T16:54:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff505600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff5000ad520 0x7ff5001644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff505600700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff5000ad520 0x7ff5001644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49752/0 (socket says 172.21.15.3:49752) 2024-09-13T16:54:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ffe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 0x7ff50006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff505600700 1 -- 172.21.15.3:0/2004379033 learned_addr learned my addr 172.21.15.3:0/2004379033 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff504c00700 1 --2- 172.21.15.3:0/2004379033 >> v2:172.21.15.157:3300/0 conn(0x7ff50006ce80 0x7ff500098ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ffe00700 1 -- 172.21.15.3:0/2004379033 >> v2:172.21.15.157:3300/0 conn(0x7ff50006ce80 msgr2=0x7ff500098ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ffe00700 1 --2- 172.21.15.3:0/2004379033 >> v2:172.21.15.157:3300/0 conn(0x7ff50006ce80 0x7ff500098ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ffe00700 1 -- 172.21.15.3:0/2004379033 >> v2:172.21.15.3:3300/0 conn(0x7ff5000ad520 msgr2=0x7ff5001644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ffe00700 1 --2- 172.21.15.3:0/2004379033 >> v2:172.21.15.3:3300/0 conn(0x7ff5000ad520 0x7ff5001644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ffe00700 1 -- 172.21.15.3:0/2004379033 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50006fc70 con 0x7ff50006d260 2024-09-13T16:54:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ffe00700 1 --2- 172.21.15.3:0/2004379033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 0x7ff50006c510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff500066d40 tx=0x7ff4ec009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f91ece2ac66b6b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ff400700 1 -- 172.21.15.3:0/2004379033 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec011830 con 0x7ff50006d260 2024-09-13T16:54:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.818+0000 7ff4ff400700 1 -- 172.21.15.3:0/2004379033 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec011990 con 0x7ff50006d260 2024-09-13T16:54:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff4ff400700 1 -- 172.21.15.3:0/2004379033 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec011d20 con 0x7ff50006d260 2024-09-13T16:54:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 -- 172.21.15.3:0/2004379033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 msgr2=0x7ff50006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 --2- 172.21.15.3:0/2004379033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 0x7ff50006c510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff500066d40 tx=0x7ff4ec009b30 comp rx=0 tx=0).stop 2024-09-13T16:54:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 -- 172.21.15.3:0/2004379033 shutdown_connections 2024-09-13T16:54:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 --2- 172.21.15.3:0/2004379033 >> v2:172.21.15.3:3300/0 conn(0x7ff5000ad520 0x7ff5001644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 --2- 172.21.15.3:0/2004379033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 0x7ff50006c510 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 --2- 172.21.15.3:0/2004379033 >> v2:172.21.15.157:3300/0 conn(0x7ff50006ce80 0x7ff500098ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 -- 172.21.15.3:0/2004379033 >> 172.21.15.3:0/2004379033 conn(0x7ff500071860 msgr2=0x7ff500071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 -- 172.21.15.3:0/2004379033 shutdown_connections 2024-09-13T16:54:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 -- 172.21.15.3:0/2004379033 wait complete. 2024-09-13T16:54:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 Processor -- start 2024-09-13T16:54:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.822+0000 7ff507600700 1 -- start start 2024-09-13T16:54:20.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff507600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50006ce80 0x7ff50015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff507600700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 0x7ff50015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff507600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000ad520 0x7ff5001583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff507600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50015b640 con 0x7ff5000ad520 2024-09-13T16:54:20.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff507600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50015b850 con 0x7ff50006ce80 2024-09-13T16:54:20.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff507600700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50015bb30 con 0x7ff50006d260 2024-09-13T16:54:20.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000ad520 0x7ff5001583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000ad520 0x7ff5001583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49754/0 (socket says 172.21.15.3:49754) 2024-09-13T16:54:20.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 -- 172.21.15.3:0/1640931278 learned_addr learned my addr 172.21.15.3:0/1640931278 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:20.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff4ffe00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 0x7ff50015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff504c00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50006ce80 0x7ff50015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 -- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 msgr2=0x7ff50015b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50006d260 0x7ff50015b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 -- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50006ce80 msgr2=0x7ff50015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50006ce80 0x7ff50015e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 -- 172.21.15.3:0/1640931278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0094b0 con 0x7ff5000ad520 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff504c00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50006ce80 0x7ff50015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff505600700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000ad520 0x7ff5001583c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800efc0 tx=0x7ff4f800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8003c40 con 0x7ff5000ad520 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f800f040 con 0x7ff5000ad520 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50015be10 con 0x7ff5000ad520 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f800c710 con 0x7ff5000ad520 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.826+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50015c3d0 con 0x7ff5000ad520 2024-09-13T16:54:20.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.830+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff50006e3a0 con 0x7ff5000ad520 2024-09-13T16:54:20.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.830+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff4f800c870 con 0x7ff5000ad520 2024-09-13T16:54:20.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff4fcc00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4e0066730 0x7ff4e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7ff4f8084fe0 con 0x7ff5000ad520 2024-09-13T16:54:20.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff4fcc00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7ff4e006cb70 0x7ff4e006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:20.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 --> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4e006f680 con 0x7ff4e006cb70 2024-09-13T16:54:20.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4f80853f0 con 0x7ff5000ad520 2024-09-13T16:54:20.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff504c00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4e0066730 0x7ff4e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff4ffe00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7ff4e006cb70 0x7ff4e006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff504c00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4e0066730 0x7ff4e0068bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009bc0 tx=0x7ff4f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.834+0000 7ff4ffe00700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7ff4e006cb70 0x7ff4e006efb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:20.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.838+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 <== osd.7 v2:172.21.15.157:6812/3325903215 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff4e006f680 con 0x7ff4e006cb70 2024-09-13T16:54:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 --> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff5000ae480 con 0x7ff4e006cb70 2024-09-13T16:54:20.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff4fcc00700 1 -- 172.21.15.3:0/1640931278 <== osd.7 v2:172.21.15.157:6812/3325903215 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff4ec022040 con 0x7ff4e006cb70 2024-09-13T16:54:20.861 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-13T16:54:20.861 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T16:54:20.861 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-13T16:54:20.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7ff4e006cb70 msgr2=0x7ff4e006efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6812/3325903215,v1:172.21.15.157:6814/3325903215] conn(0x7ff4e006cb70 0x7ff4e006efb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4e0066730 msgr2=0x7ff4e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7ff4e0066730 0x7ff4e0068bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009bc0 tx=0x7ff4f4008040 comp rx=0 tx=0).stop 2024-09-13T16:54:20.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000ad520 msgr2=0x7ff5001583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:20.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 --2- 172.21.15.3:0/1640931278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000ad520 0x7ff5001583c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800efc0 tx=0x7ff4f800c0c0 comp rx=0 tx=0).stop 2024-09-13T16:54:20.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff505600700 1 -- 172.21.15.3:0/1640931278 reap_dead start 2024-09-13T16:54:20.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 shutdown_connections 2024-09-13T16:54:20.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 >> 172.21.15.3:0/1640931278 conn(0x7ff500071860 msgr2=0x7ff500160e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:20.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 shutdown_connections 2024-09-13T16:54:20.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:20.858+0000 7ff507600700 1 -- 172.21.15.3:0/1640931278 wait complete. 2024-09-13T16:54:20.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T16:54:21.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.054+0000 7fb7d7200700 1 Processor -- start 2024-09-13T16:54:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.054+0000 7fb7d7200700 1 -- start start 2024-09-13T16:54:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.058+0000 7fb7d7200700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7d80b3f40 0x7fb7d80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.058+0000 7fb7d7200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d806f8e0 0x7fb7d806fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.058+0000 7fb7d7200700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d815e070 con 0x7fb7d815d4d0 2024-09-13T16:54:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.058+0000 7fb7d7200700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d815e2a0 con 0x7fb7d80b3f40 2024-09-13T16:54:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.058+0000 7fb7d7200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8070270 con 0x7fb7d806f8e0 2024-09-13T16:54:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d806f8e0 0x7fb7d806fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5400700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d806f8e0 0x7fb7d806fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.3:45166/0 (socket says 172.21.15.3:45166) 2024-09-13T16:54:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5400700 1 -- 172.21.15.3:0/1689465601 learned_addr learned my addr 172.21.15.3:0/1689465601 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5e00700 1 --2- 172.21.15.3:0/1689465601 >> v2:172.21.15.157:3300/0 conn(0x7fb7d80b3f40 0x7fb7d80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5400700 1 -- 172.21.15.3:0/1689465601 >> v2:172.21.15.157:3300/0 conn(0x7fb7d80b3f40 msgr2=0x7fb7d80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5400700 1 --2- 172.21.15.3:0/1689465601 >> v2:172.21.15.157:3300/0 conn(0x7fb7d80b3f40 0x7fb7d80b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5400700 1 -- 172.21.15.3:0/1689465601 >> v1:172.21.15.157:6789/0 conn(0x7fb7d815d4d0 legacy=0x7fb7d815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5400700 1 -- 172.21.15.3:0/1689465601 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d81617e0 con 0x7fb7d806f8e0 2024-09-13T16:54:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d5400700 1 --2- 172.21.15.3:0/1689465601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d806f8e0 0x7fb7d806fd30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009910 tx=0x7fb7c4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79b23e4f63958b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d4a00700 1 -- 172.21.15.3:0/1689465601 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4011830 con 0x7fb7d806f8e0 2024-09-13T16:54:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d4a00700 1 -- 172.21.15.3:0/1689465601 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c4011990 con 0x7fb7d806f8e0 2024-09-13T16:54:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d4a00700 1 -- 172.21.15.3:0/1689465601 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4011d40 con 0x7fb7d806f8e0 2024-09-13T16:54:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d7200700 1 -- 172.21.15.3:0/1689465601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d806f8e0 msgr2=0x7fb7d806fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d7200700 1 --2- 172.21.15.3:0/1689465601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d806f8e0 0x7fb7d806fd30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009910 tx=0x7fb7c4009c20 comp rx=0 tx=0).stop 2024-09-13T16:54:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d7200700 1 -- 172.21.15.3:0/1689465601 shutdown_connections 2024-09-13T16:54:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d7200700 1 --2- 172.21.15.3:0/1689465601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d806f8e0 0x7fb7d806fd30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d7200700 1 --2- 172.21.15.3:0/1689465601 >> v2:172.21.15.157:3300/0 conn(0x7fb7d80b3f40 0x7fb7d80b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d7200700 1 -- 172.21.15.3:0/1689465601 >> 172.21.15.3:0/1689465601 conn(0x7fb7d80b2910 msgr2=0x7fb7d80b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d7200700 1 -- 172.21.15.3:0/1689465601 shutdown_connections 2024-09-13T16:54:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.062+0000 7fb7d7200700 1 -- 172.21.15.3:0/1689465601 wait complete. 2024-09-13T16:54:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.066+0000 7fb7d7200700 1 Processor -- start 2024-09-13T16:54:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.066+0000 7fb7d7200700 1 -- start start 2024-09-13T16:54:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d7200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7d806f8e0 0x7fb7d8144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b3f40 0x7fb7d8143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d7200700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815d4d0 0x7fb7d81439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d7200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8145490 con 0x7fb7d806f8e0 2024-09-13T16:54:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8145770 con 0x7fb7d80b3f40 2024-09-13T16:54:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d7200700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8140410 con 0x7fb7d815d4d0 2024-09-13T16:54:21.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7d806f8e0 0x7fb7d8144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7d806f8e0 0x7fb7d8144ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49756/0 (socket says 172.21.15.3:49756) 2024-09-13T16:54:21.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 -- 172.21.15.3:0/579433622 learned_addr learned my addr 172.21.15.3:0/579433622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:21.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d6800700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815d4d0 0x7fb7d81439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5400700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b3f40 0x7fb7d8143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 -- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815d4d0 msgr2=0x7fb7d81439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815d4d0 0x7fb7d81439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 -- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b3f40 msgr2=0x7fb7d8143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b3f40 0x7fb7d8143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 -- 172.21.15.3:0/579433622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4009570 con 0x7fb7d806f8e0 2024-09-13T16:54:21.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5400700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b3f40 0x7fb7d8143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:21.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.070+0000 7fb7d5e00700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7d806f8e0 0x7fb7d8144ad0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800e9d0 tx=0x7fb7c800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:21.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.074+0000 7fb7ce000700 1 -- 172.21.15.3:0/579433622 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8009720 con 0x7fb7d806f8e0 2024-09-13T16:54:21.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.074+0000 7fb7ce000700 1 -- 172.21.15.3:0/579433622 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c800ee70 con 0x7fb7d806f8e0 2024-09-13T16:54:21.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.074+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d81406f0 con 0x7fb7d806f8e0 2024-09-13T16:54:21.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.074+0000 7fb7ce000700 1 -- 172.21.15.3:0/579433622 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800ca50 con 0x7fb7d806f8e0 2024-09-13T16:54:21.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.074+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d8145a50 con 0x7fb7d806f8e0 2024-09-13T16:54:21.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.074+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d806fd30 con 0x7fb7d806f8e0 2024-09-13T16:54:21.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.078+0000 7fb7ce000700 1 -- 172.21.15.3:0/579433622 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb7c800cbb0 con 0x7fb7d806f8e0 2024-09-13T16:54:21.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.078+0000 7fb7ce000700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7b8066730 0x7fb7b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.078+0000 7fb7ce000700 1 -- 172.21.15.3:0/579433622 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fb7c80840b0 con 0x7fb7d806f8e0 2024-09-13T16:54:21.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.078+0000 7fb7d5400700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7b8066730 0x7fb7b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.082+0000 7fb7d5400700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7b8066730 0x7fb7b8068bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4000c00 tx=0x7fb7c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:21.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.086+0000 7fb7ce000700 1 -- 172.21.15.3:0/579433622 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb7c8051520 con 0x7fb7d806f8e0 2024-09-13T16:54:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.266+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7d8160fe0 con 0x7fb7b8066730 2024-09-13T16:54:21.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.266+0000 7fb7ce000700 1 -- 172.21.15.3:0/579433622 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb7c4011b80 con 0x7fb7b8066730 2024-09-13T16:54:21.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7b8066730 msgr2=0x7fb7b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7b8066730 0x7fb7b8068bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4000c00 tx=0x7fb7c4008030 comp rx=0 tx=0).stop 2024-09-13T16:54:21.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7d806f8e0 msgr2=0x7fb7d8144ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7d806f8e0 0x7fb7d8144ad0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800e9d0 tx=0x7fb7c800ece0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 shutdown_connections 2024-09-13T16:54:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb7b8066730 0x7fb7b8068bf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815d4d0 0x7fb7d81439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b3f40 0x7fb7d8143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 --2- 172.21.15.3:0/579433622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7d806f8e0 0x7fb7d8144ad0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 >> 172.21.15.3:0/579433622 conn(0x7fb7d80b2910 msgr2=0x7fb7d815ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 shutdown_connections 2024-09-13T16:54:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.270+0000 7fb7d7200700 1 -- 172.21.15.3:0/579433622 wait complete. 2024-09-13T16:54:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T16:54:21.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:54:21.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.470+0000 7f3a22000700 1 Processor -- start 2024-09-13T16:54:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.474+0000 7f3a22000700 1 -- start start 2024-09-13T16:54:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.474+0000 7f3a22000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a1c1443b0 0x7f3a1c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.474+0000 7f3a22000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c06ccf0 0x7f3a1c15b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.474+0000 7f3a22000700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c0ce8e0 con 0x7f3a1c144790 2024-09-13T16:54:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.474+0000 7f3a22000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c0ccd30 con 0x7f3a1c1443b0 2024-09-13T16:54:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.474+0000 7f3a22000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0cd010 con 0x7f3a1c06ccf0 2024-09-13T16:54:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a20c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a1c1443b0 0x7f3a1c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a20c00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a1c1443b0 0x7f3a1c0b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49760/0 (socket says 172.21.15.3:49760) 2024-09-13T16:54:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a20c00700 1 -- 172.21.15.3:0/2638309341 learned_addr learned my addr 172.21.15.3:0/2638309341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a21600700 1 --2- 172.21.15.3:0/2638309341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c06ccf0 0x7f3a1c15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a20c00700 1 -- 172.21.15.3:0/2638309341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c06ccf0 msgr2=0x7f3a1c15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a20c00700 1 --2- 172.21.15.3:0/2638309341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c06ccf0 0x7f3a1c15b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a20c00700 1 -- 172.21.15.3:0/2638309341 >> v1:172.21.15.3:6789/0 conn(0x7f3a1c144790 legacy=0x7f3a1c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a1be00700 1 -- 172.21.15.3:0/2638309341 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293407555 0 0) 0x7f3a1c0ce8e0 con 0x7f3a1c144790 2024-09-13T16:54:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a20c00700 1 -- 172.21.15.3:0/2638309341 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0cd2f0 con 0x7f3a1c1443b0 2024-09-13T16:54:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a21600700 1 --2- 172.21.15.3:0/2638309341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c06ccf0 0x7f3a1c15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a20c00700 1 --2- 172.21.15.3:0/2638309341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c0b39f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009910 tx=0x7f3a0c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1c2c5228d3cb62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a1be00700 1 -- 172.21.15.3:0/2638309341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00ba60 con 0x7f3a1c1443b0 2024-09-13T16:54:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a1be00700 1 -- 172.21.15.3:0/2638309341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c00bbc0 con 0x7f3a1c1443b0 2024-09-13T16:54:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a1be00700 1 -- 172.21.15.3:0/2638309341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c009e60 con 0x7f3a1c1443b0 2024-09-13T16:54:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a22000700 1 -- 172.21.15.3:0/2638309341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 msgr2=0x7f3a1c0b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.478+0000 7f3a22000700 1 --2- 172.21.15.3:0/2638309341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c0b39f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009910 tx=0x7f3a0c009c20 comp rx=0 tx=0).stop 2024-09-13T16:54:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.482+0000 7f3a22000700 1 -- 172.21.15.3:0/2638309341 shutdown_connections 2024-09-13T16:54:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.482+0000 7f3a22000700 1 --2- 172.21.15.3:0/2638309341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c06ccf0 0x7f3a1c15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.482+0000 7f3a22000700 1 --2- 172.21.15.3:0/2638309341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c0b39f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.482+0000 7f3a22000700 1 -- 172.21.15.3:0/2638309341 >> 172.21.15.3:0/2638309341 conn(0x7f3a1c0b0cf0 msgr2=0x7f3a1c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.482+0000 7f3a22000700 1 -- 172.21.15.3:0/2638309341 shutdown_connections 2024-09-13T16:54:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.482+0000 7f3a22000700 1 -- 172.21.15.3:0/2638309341 wait complete. 2024-09-13T16:54:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.482+0000 7f3a22000700 1 Processor -- start 2024-09-13T16:54:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 -- start start 2024-09-13T16:54:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a1c06ccf0 0x7f3a1c13e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c13ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c13e8f0 0x7f3a1c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0a8d10 con 0x7f3a1c1443b0 2024-09-13T16:54:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1f1b10 con 0x7f3a1c06ccf0 2024-09-13T16:54:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1f1dc0 con 0x7f3a1c13e8f0 2024-09-13T16:54:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c13ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c13ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49768/0 (socket says 172.21.15.3:49768) 2024-09-13T16:54:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 -- 172.21.15.3:0/2887962306 learned_addr learned my addr 172.21.15.3:0/2887962306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a21600700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c13e8f0 0x7f3a1c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a20c00700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a1c06ccf0 0x7f3a1c13e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 -- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c13e8f0 msgr2=0x7f3a1c1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c13e8f0 0x7f3a1c1571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 -- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a1c06ccf0 msgr2=0x7f3a1c13e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a1c06ccf0 0x7f3a1c13e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 -- 172.21.15.3:0/2887962306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c009570 con 0x7f3a1c1443b0 2024-09-13T16:54:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a13e00700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c13ad10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400c520 tx=0x7f3a0400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a19600700 1 -- 172.21.15.3:0/2887962306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040054e0 con 0x7f3a1c1443b0 2024-09-13T16:54:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a19600700 1 -- 172.21.15.3:0/2887962306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a040036a0 con 0x7f3a1c1443b0 2024-09-13T16:54:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c1f2100 con 0x7f3a1c1443b0 2024-09-13T16:54:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a19600700 1 -- 172.21.15.3:0/2887962306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04003a50 con 0x7f3a1c1443b0 2024-09-13T16:54:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.486+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c13b250 con 0x7f3a1c1443b0 2024-09-13T16:54:21.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.490+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c1f1f30 con 0x7f3a1c1443b0 2024-09-13T16:54:21.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.490+0000 7f3a19600700 1 -- 172.21.15.3:0/2887962306 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a04003bb0 con 0x7f3a1c1443b0 2024-09-13T16:54:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.494+0000 7f3a19600700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f39f8066730 0x7f39f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.494+0000 7f3a19600700 1 -- 172.21.15.3:0/2887962306 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f3a040834e0 con 0x7f3a1c1443b0 2024-09-13T16:54:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.494+0000 7f3a20c00700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f39f8066730 0x7f39f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.494+0000 7f3a20c00700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f39f8066730 0x7f39f8068bf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c141d80 tx=0x7f3a0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.502+0000 7f3a19600700 1 -- 172.21.15.3:0/2887962306 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3a04050950 con 0x7f3a1c1443b0 2024-09-13T16:54:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.698+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a1c0cca20 con 0x7f39f8066730 2024-09-13T16:54:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.702+0000 7f3a19600700 1 -- 172.21.15.3:0/2887962306 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48519 (secure 0 0 0) 0x7f3a0c00b860 con 0x7f39f8066730 2024-09-13T16:54:21.708 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f39f8066730 msgr2=0x7f39f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f39f8066730 0x7f39f8068bf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c141d80 tx=0x7f3a0c008030 comp rx=0 tx=0).stop 2024-09-13T16:54:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 msgr2=0x7f3a1c13ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c13ad10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400c520 tx=0x7f3a0400c830 comp rx=0 tx=0).stop 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 shutdown_connections 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f39f8066730 0x7f39f8068bf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a1c13e8f0 0x7f3a1c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a1c1443b0 0x7f3a1c13ad10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 --2- 172.21.15.3:0/2887962306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a1c06ccf0 0x7f3a1c13e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 >> 172.21.15.3:0/2887962306 conn(0x7f3a1c0b0cf0 msgr2=0x7f3a1c1432c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 shutdown_connections 2024-09-13T16:54:21.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:21.706+0000 7f3a22000700 1 -- 172.21.15.3:0/2887962306 wait complete. 2024-09-13T16:54:21.713 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:54:21.734 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-13T16:54:21.279860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":248248,"kb_used_data":7496,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726472,"statfs":{"total":773094113280,"available":772839907328,"internally_reserved":0,"allocated":7675904,"data_stored":4141152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003460"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T16:54:07.794569+0000","last_change":"2024-09-13T16:50:21.360237+0000","last_active":"2024-09-13T16:54:07.794569+0000","last_peered":"2024-09-13T16:54:07.794569+0000","last_clean":"2024-09-13T16:54:07.794569+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:54:07.794569+0000","last_undegraded":"2024-09-13T16:54:07.794569+0000","last_fullsized":"2024-09-13T16:54:07.794569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:50:19.989947+0000","last_clean_scrub_stamp":"2024-09-13T16:50:19.989947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:16.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1404928,"data_stored":960420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.103,"15min":1.137},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4339999999999999,"5min":1.585,"15min":1.585},"last":1.3},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.0860000000000001},"min":{"1min":0.78500000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5209999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.359}]},{"osd":1,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.238,"15min":1.266},"min":{"1min":0.89500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.284,"15min":1.29},"min":{"1min":0.90300000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.696,"5min":1.897,"15min":1.897},"last":1.454}]},{"osd":2,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.337,"15min":1.377},"min":{"1min":1.0289999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.7569999999999999,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2430000000000001,"15min":1.3069999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.468,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.1659999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2230000000000001,"15min":1.2589999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.4570000000000001,"5min":2.161,"15min":2.161},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2829999999999999},"min":{"1min":0.86399999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.6100000000000001,"5min":1.704,"15min":1.704},"last":1.544}]},{"osd":4,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.165,"15min":1.153},"min":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.329,"15min":1.3360000000000001},"min":{"1min":0.88200000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7110000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.3839999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.361,"15min":1.363},"min":{"1min":0.71299999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.806,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.597},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.341,"15min":1.3440000000000001},"min":{"1min":0.85399999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.867,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.4199999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:22 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.371,"15min":1.3680000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.704,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4039999999999999,"15min":1.4219999999999999},"min":{"1min":0.98899999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.796,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.502}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.1279999999999999},"min":{"1min":0.74299999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.129,"5min":1.222,"15min":1.3},"min":{"1min":0.76000000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.526,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":1.1910000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.232,"15min":1.3120000000000001},"min":{"1min":0.70099999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3740000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.206},"min":{"1min":0.88600000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5569999999999999,"5min":1.623,"15min":1.623},"last":0.67600000000000005}]},{"osd":2,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.28,"15min":1.323},"min":{"1min":0.85299999999999998,"5min":0.745,"15min":0.745},"max":{"1min":1.661,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2509999999999999,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.462,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.3600000000000001,"15min":1.3999999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6259999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.395,"15min":1.45},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.6040000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.302}]},{"osd":4,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.2330000000000001,"15min":1.2689999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4450000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.264,"15min":1.306},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4990000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.077}]},{"osd":5,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.266,"15min":1.3149999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6639999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.383,"15min":1.431},"min":{"1min":0.89100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6779999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.2350000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:53:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4770000000000001,"15min":1.5249999999999999},"min":{"1min":1.03,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.7110000000000001,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.526},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4430000000000001,"15min":1.5069999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8280000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.4570000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31912,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.1910000000000001,"15min":1.1850000000000001},"min":{"1min":0.745,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5720000000000001,"5min":2.222,"15min":2.222},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.099},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":1.601,"15min":1.601},"last":1.105}]},{"osd":1,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.395,"15min":1.4399999999999999},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.6080000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.583,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.53}]},{"osd":2,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.248,"15min":1.2649999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.55,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.252,"15min":1.264},"min":{"1min":0.96199999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.611,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.327}]},{"osd":3,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.3480000000000001,"15min":1.4199999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5509999999999999,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.008},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3180000000000001,"15min":1.3360000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.643,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.266}]},{"osd":4,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.198,"15min":1.21},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4319999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.3180000000000001,"15min":1.3380000000000001},"min":{"1min":0.89300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6890000000000001,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.3029999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2829999999999999,"15min":1.3220000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.6759999999999999,"5min":1.887,"15min":1.887},"last":1.411},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.284,"15min":1.2749999999999999},"min":{"1min":0.90100000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.657,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.149}]},{"osd":7,"last update":"Fri Sep 13 16:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.4690000000000001,"15min":1.5029999999999999},"min":{"1min":0.95699999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7549999999999999,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.21},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.474},"min":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.7410000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.175}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0760000000000001,"15min":1.0980000000000001},"min":{"1min":0.75900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3899999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.169,"15min":1.2390000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4590000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2470000000000001,"15min":1.29},"min":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6699999999999999,"5min":1.919,"15min":1.919},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1659999999999999,"15min":1.196},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.361,"5min":1.99,"15min":1.99},"last":0.995}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.27,"15min":1.3169999999999999},"min":{"1min":0.83499999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7070000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.1850000000000001,"15min":1.2250000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4059999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.367,"15min":1.4359999999999999},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.444,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.4059999999999999,"15min":1.4750000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":1.4930000000000001,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.0820000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.2529999999999999,"15min":1.3320000000000001},"min":{"1min":0.63500000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.149},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3320000000000001,"15min":1.385},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.754,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.3440000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.423,"15min":1.5069999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.556,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.4990000000000001,"15min":1.601},"min":{"1min":0.74099999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.532,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.381}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4730000000000001,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5960000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7849999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.3080000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95999999999999996,"15min":1.006},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4410000000000001,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2589999999999999,"5min":1.661,"15min":1.661},"last":0.72699999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.996,"15min":1.0429999999999999},"min":{"1min":0.54200000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.663,"5min":1.931,"15min":1.931},"last":0.876},{"interface":"front","average":{"1min":1.03,"5min":1.1399999999999999,"15min":1.1839999999999999},"min":{"1min":0.73899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.415,"5min":1.849,"15min":1.849},"last":1.276}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.246,"15min":1.319},"min":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5640000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.3140000000000001,"15min":1.385},"min":{"1min":0.76200000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.607,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.196}]},{"osd":4,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5109999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.431,"15min":1.498},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.528,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.321}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.5349999999999999,"15min":1.621},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.8180000000000001,"5min":2.79,"15min":2.79},"last":1.226},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.516,"15min":1.611},"min":{"1min":0.96799999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5409999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.373}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.583,"15min":1.6759999999999999},"min":{"1min":1.0840000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.613,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.51,"15min":1.603},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.627,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.2549999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.579,"15min":1.661},"min":{"1min":0.94799999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.6990000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.629,"15min":1.724},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7629999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.411}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.175,"5min":1.925,"15min":1.925},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.081,"15min":1.119},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3140000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.1319999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.92500000000000004,"15min":0.96299999999999997},"min":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.0369999999999999,"15min":1.073},"min":{"1min":0.57699999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.349,"5min":2.028,"15min":2.028},"last":1.3819999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.72499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6359999999999999,"5min":1.948,"15min":1.948},"last":1.25},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.141,"15min":1.139},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.1060000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.3169999999999999},"min":{"1min":0.81599999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.577,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.3999999999999999,"15min":1.4530000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5580000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.6100000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.5,"15min":1.5680000000000001},"min":{"1min":0.93799999999999994,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3799999999999999,"15min":1.4139999999999999},"min":{"1min":0.83699999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.647,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.325}]},{"osd":6,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.5069999999999999,"15min":1.5609999999999999},"min":{"1min":1.044,"5min":0.753,"15min":0.753},"max":{"1min":1.675,"5min":2.625,"15min":2.625},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.522,"15min":1.5900000000000001},"min":{"1min":0.91400000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.742,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.4670000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5800000000000001,"15min":1.6579999999999999},"min":{"1min":0.96899999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.681,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.538},{"interface":"front","average":{"1min":1.458,"5min":1.597,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.872,"15min":0.872},"max":{"1min":1.722,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.5149999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.61699999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2170000000000001,"5min":1.804,"15min":1.804},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.034,"5min":0.998,"15min":1},"min":{"1min":0.70699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4419999999999999,"5min":1.954,"15min":1.954},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.042,"15min":1.0700000000000001},"min":{"1min":0.58599999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5700000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2,"15min":1.177},"min":{"1min":0.70399999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.804,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1439999999999999,"15min":1.153},"min":{"1min":0.74099999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.869,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.5409999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.284,"15min":1.3140000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.752,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.244},{"interface":"front","average":{"1min":1.365,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":1.1539999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.8300000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.91700000000000004}]},{"osd":5,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.5,"15min":1.528},"min":{"1min":0.99299999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.0779999999999998,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4890000000000001,"15min":1.5269999999999999},"min":{"1min":1.1899999999999999,"5min":1,"15min":1},"max":{"1min":1.843,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.296}]},{"osd":6,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.504,"15min":1.536},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.3239999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.306},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.452,"15min":1.494},"min":{"1min":1.141,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.7310000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.113}]},{"osd":7,"last update":"Fri Sep 13 16:53:23 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.52,"15min":1.55},"min":{"1min":1.1899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":1.758,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.167},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.6020000000000001},"min":{"1min":1.1180000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.4390000000000001,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.202}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1404928,"data_stored":960420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.90200000000000002,"15min":0.94599999999999995},"min":{"1min":0.55600000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.2809999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.73799999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5029999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3340000000000001,"5min":1.482,"15min":1.482},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.129,"15min":1.153},"min":{"1min":0.69399999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4219999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.01,"15min":1.0269999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2749999999999999,"5min":1.653,"15min":1.653},"last":1.139},{"interface":"front","average":{"1min":1.149,"5min":1.2070000000000001,"15min":1.2330000000000001},"min":{"1min":0.73599999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.631,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.319}]},{"osd":4,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.327,"15min":1.385},"min":{"1min":0.86899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.6639999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.369},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3460000000000001,"15min":1.361},"min":{"1min":0.94899999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.556,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.409}]},{"osd":5,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.446,"15min":1.498},"min":{"1min":1.1160000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8080000000000001,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":1.343},{"interface":"front","average":{"1min":1.306,"5min":1.4139999999999999,"15min":1.468},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.7689999999999999,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.286}]},{"osd":6,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4910000000000001,"15min":1.5429999999999999},"min":{"1min":0.97099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7370000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.488},{"interface":"front","average":{"1min":1.46,"5min":1.5289999999999999,"15min":1.583},"min":{"1min":1.1719999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.806,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":1.165}]},{"osd":7,"last update":"Fri Sep 13 16:53:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.5620000000000001,"15min":1.6220000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":1.9019999999999999,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.25},{"interface":"front","average":{"1min":1.383,"5min":1.4890000000000001,"15min":1.536},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.7709999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.4339999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:54:21.738 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T16:50:19.989947+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=50, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=54, tm_sec=14, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T16:54:21.738 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T16:54:29.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:54:29.082 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:54:41.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:54:41.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.914+0000 7f2081800700 1 Processor -- start 2024-09-13T16:54:41.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f2081800700 1 -- start start 2024-09-13T16:54:41.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f2081800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f207c071240 0x7f207c06cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:41.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f2081800700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207c0716f0 con 0x7f207c070db0 2024-09-13T16:54:41.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f2081800700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207c0b3450 con 0x7f207c15b660 2024-09-13T16:54:41.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f2081800700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c0b40d0 con 0x7f207c071240 2024-09-13T16:54:41.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f207a000700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f207c070db0 0x7f207c06c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33442/0 (socket says 172.21.15.3:33442) 2024-09-13T16:54:41.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f207a000700 1 -- 172.21.15.3:0/352038024 learned_addr learned my addr 172.21.15.3:0/352038024 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:41.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f207b400700 1 --2- 172.21.15.3:0/352038024 >> v2:172.21.15.3:3300/0 conn(0x7f207c071240 0x7f207c06cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:41.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.918+0000 7f2079600700 1 -- 172.21.15.3:0/352038024 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792064995 0 0) 0x7f207c0716f0 con 0x7f207c070db0 2024-09-13T16:54:41.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2079600700 1 -- 172.21.15.3:0/352038024 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20600035d0 con 0x7f207c070db0 2024-09-13T16:54:41.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2079600700 1 -- 172.21.15.3:0/352038024 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924592861 0 0) 0x7f207c0b3450 con 0x7f207c15b660 2024-09-13T16:54:41.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2079600700 1 -- 172.21.15.3:0/352038024 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f207c0716f0 con 0x7f207c15b660 2024-09-13T16:54:41.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f207b400700 1 -- 172.21.15.3:0/352038024 >> v1:172.21.15.157:6789/0 conn(0x7f207c15b660 legacy=0x7f207c0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:41.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f207b400700 1 -- 172.21.15.3:0/352038024 >> v1:172.21.15.3:6789/0 conn(0x7f207c070db0 legacy=0x7f207c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:41.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f207b400700 1 -- 172.21.15.3:0/352038024 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0b3730 con 0x7f207c071240 2024-09-13T16:54:41.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f207b400700 1 --2- 172.21.15.3:0/352038024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c071240 0x7f207c06cc50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2074009850 tx=0x7f2074009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5be857d0784a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:41.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2079600700 1 -- 172.21.15.3:0/352038024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074011830 con 0x7f207c071240 2024-09-13T16:54:41.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2079600700 1 -- 172.21.15.3:0/352038024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2074011990 con 0x7f207c071240 2024-09-13T16:54:41.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2079600700 1 -- 172.21.15.3:0/352038024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074011d40 con 0x7f207c071240 2024-09-13T16:54:41.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2081800700 1 -- 172.21.15.3:0/352038024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c071240 msgr2=0x7f207c06cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:41.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2081800700 1 --2- 172.21.15.3:0/352038024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c071240 0x7f207c06cc50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2074009850 tx=0x7f2074009b60 comp rx=0 tx=0).stop 2024-09-13T16:54:41.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2081800700 1 -- 172.21.15.3:0/352038024 shutdown_connections 2024-09-13T16:54:41.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2081800700 1 --2- 172.21.15.3:0/352038024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c071240 0x7f207c06cc50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:41.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2081800700 1 -- 172.21.15.3:0/352038024 >> 172.21.15.3:0/352038024 conn(0x7f207c06a390 msgr2=0x7f207c06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:41.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2081800700 1 -- 172.21.15.3:0/352038024 shutdown_connections 2024-09-13T16:54:41.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.922+0000 7f2081800700 1 -- 172.21.15.3:0/352038024 wait complete. 2024-09-13T16:54:41.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.926+0000 7f2081800700 1 Processor -- start 2024-09-13T16:54:41.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.926+0000 7f2081800700 1 -- start start 2024-09-13T16:54:41.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2081800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c070db0 0x7f207c15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:41.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2081800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207c071240 0x7f207c1655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2081800700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f207c15b660 0x7f207c1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2081800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c165b20 con 0x7f207c070db0 2024-09-13T16:54:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2081800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c160f20 con 0x7f207c071240 2024-09-13T16:54:41.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2081800700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c162190 con 0x7f207c15b660 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c070db0 0x7f207c15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c070db0 0x7f207c15bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40708/0 (socket says 172.21.15.3:40708) 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 -- 172.21.15.3:0/4247189421 learned_addr learned my addr 172.21.15.3:0/4247189421 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207a000700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207c071240 0x7f207c1655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207b400700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f207c15b660 0x7f207c1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 -- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f207c15b660 msgr2=0x7f207c1609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f207c15b660 0x7f207c1609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 -- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207c071240 msgr2=0x7f207c1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:41.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207c071240 0x7f207c1655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:41.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 -- 172.21.15.3:0/4247189421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740094b0 con 0x7f207c070db0 2024-09-13T16:54:41.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207a000700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207c071240 0x7f207c1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:41.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f207aa00700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c070db0 0x7f207c15bc50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f206c001f00 tx=0x7f206c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:41.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2072a00700 1 -- 172.21.15.3:0/4247189421 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c003c20 con 0x7f207c070db0 2024-09-13T16:54:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2072a00700 1 -- 172.21.15.3:0/4247189421 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c003d80 con 0x7f207c070db0 2024-09-13T16:54:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207c143ce0 con 0x7f207c070db0 2024-09-13T16:54:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2072a00700 1 -- 172.21.15.3:0/4247189421 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c00abb0 con 0x7f207c070db0 2024-09-13T16:54:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.930+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207c1623e0 con 0x7f207c070db0 2024-09-13T16:54:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.934+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c15ba40 con 0x7f207c070db0 2024-09-13T16:54:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.934+0000 7f2072a00700 1 -- 172.21.15.3:0/4247189421 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f206c00ad10 con 0x7f207c070db0 2024-09-13T16:54:41.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.938+0000 7f2072a00700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f205c066730 0x7f205c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:41.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.938+0000 7f2072a00700 1 -- 172.21.15.3:0/4247189421 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f206c0841c0 con 0x7f207c070db0 2024-09-13T16:54:41.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.938+0000 7f207a000700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f205c066730 0x7f205c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:41.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.938+0000 7f207a000700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f205c066730 0x7f205c068bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f207c0ce2e0 tx=0x7f2068007e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:41.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:41.946+0000 7f2072a00700 1 -- 172.21.15.3:0/4247189421 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f206c0515b0 con 0x7f207c070db0 2024-09-13T16:54:42.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.134+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 --> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f207c0b3110 con 0x7f205c066730 2024-09-13T16:54:42.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.142+0000 7f2072a00700 1 -- 172.21.15.3:0/4247189421 <== mgr.4104 v2:172.21.15.157:6832/3433838649 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48526 (secure 0 0 0) 0x7f2068007100 con 0x7f205c066730 2024-09-13T16:54:42.146 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f205c066730 msgr2=0x7f205c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f205c066730 0x7f205c068bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f207c0ce2e0 tx=0x7f2068007e00 comp rx=0 tx=0).stop 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c070db0 msgr2=0x7f207c15bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c070db0 0x7f207c15bc50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f206c001f00 tx=0x7f206c00a060 comp rx=0 tx=0).stop 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 shutdown_connections 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f205c066730 0x7f205c068bf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f207c15b660 0x7f207c1609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f207c071240 0x7f207c1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 --2- 172.21.15.3:0/4247189421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f207c070db0 0x7f207c15bc50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 >> 172.21.15.3:0/4247189421 conn(0x7f207c06a390 msgr2=0x7f207c0cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 shutdown_connections 2024-09-13T16:54:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.146+0000 7f2081800700 1 -- 172.21.15.3:0/4247189421 wait complete. 2024-09-13T16:54:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-13T16:54:42.175 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-13T16:54:41.285688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":248232,"kb_used_data":7496,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726488,"statfs":{"total":773094113280,"available":772839923712,"internally_reserved":0,"allocated":7675904,"data_stored":4141152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003530"},"pg_stats":[{"pgid":"1.0","version":"67'119","reported_seq":215,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T16:54:26.101185+0000","last_change":"2024-09-13T16:54:26.101366+0000","last_active":"2024-09-13T16:54:26.101185+0000","last_peered":"2024-09-13T16:54:26.101185+0000","last_clean":"2024-09-13T16:54:26.101185+0000","last_became_active":"2024-09-13T16:50:21.358602+0000","last_became_peered":"2024-09-13T16:50:21.358602+0000","last_unstale":"2024-09-13T16:54:26.101185+0000","last_undegraded":"2024-09-13T16:54:26.101185+0000","last_fullsized":"2024-09-13T16:54:26.101185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'119","last_scrub_stamp":"2024-09-13T16:54:26.101135+0000","last_deep_scrub":"67'119","last_deep_scrub_stamp":"2024-09-13T16:54:26.101135+0000","last_clean_scrub_stamp":"2024-09-13T16:54:26.101135+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:08.337343+0000","scrub_duration":0.0096287930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1404928,"data_stored":960420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:54:23 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.0880000000000001,"15min":1.1319999999999999},"min":{"1min":0.80000000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4870000000000001,"5min":1.585,"15min":1.585},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0980000000000001,"15min":1.0880000000000001},"min":{"1min":0.75800000000000001,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.1040000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:54:23 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2170000000000001,"15min":1.2589999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5429999999999999,"5min":1.788,"15min":1.788},"last":1.282},{"interface":"front","average":{"1min":1.258,"5min":1.2769999999999999,"15min":1.2869999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.4550000000000001,"5min":1.897,"15min":1.897},"last":1.2549999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:54:23 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2829999999999999,"15min":1.359},"min":{"1min":0.84999999999999998,"5min":0.754,"15min":0.754},"max":{"1min":1.355,"5min":5.6959999999999997,"15min":5.6959999999999997},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2090000000000001,"15min":1.2949999999999999},"min":{"1min":0.89300000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.4950000000000001,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":1.2150000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:54:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1970000000000001,"15min":1.25},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5129999999999999,"5min":2.161,"15min":2.161},"last":1.143},{"interface":"front","average":{"1min":1.321,"5min":1.2949999999999999,"15min":1.286},"min":{"1min":1.01,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5740000000000001,"5min":1.704,"15min":1.704},"last":1.3720000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:54:23 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1830000000000001,"15min":1.159},"min":{"1min":0.83099999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6639999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.331,"15min":1.337},"min":{"1min":0.97099999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Sep 13 16:54:23 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.353,"15min":1.361},"min":{"1min":0.91700000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6339999999999999,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3360000000000001,"15min":1.3420000000000001},"min":{"1min":0.86699999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5760000000000001,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.335}]},{"osd":6,"last update":"Fri Sep 13 16:54:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3680000000000001,"15min":1.367},"min":{"1min":1.006,"5min":1.006,"15min":1.006},"max":{"1min":1.619,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3919999999999999,"15min":1.4179999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.6659999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":0.94899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.091,"15min":1.125},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.52,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.875},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.169,"15min":1.282},"min":{"1min":0.75,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.464,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":1.151}]},{"osd":1,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.171,"15min":1.2909999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.411,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.054},{"interface":"front","average":{"1min":1.079,"5min":1.149,"15min":1.1970000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.387,"5min":1.623,"15min":1.623},"last":0.98199999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2430000000000001,"15min":1.3109999999999999},"min":{"1min":0.84099999999999997,"5min":0.745,"15min":0.745},"max":{"1min":1.401,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.071},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.218,"15min":1.28},"min":{"1min":0.84899999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.3859999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.92300000000000004}]},{"osd":3,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3140000000000001,"15min":1.385},"min":{"1min":0.90900000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.504,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.177},{"interface":"front","average":{"1min":1.181,"5min":1.335,"15min":1.4299999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.512,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.2250000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1899999999999999,"15min":1.2549999999999999},"min":{"1min":0.68100000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5589999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2110000000000001,"15min":1.288},"min":{"1min":0.81699999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4319999999999999,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.94299999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.2150000000000001,"15min":1.298},"min":{"1min":0.86199999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.3879999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3420000000000001,"15min":1.417},"min":{"1min":0.998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.536,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.2609999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.4299999999999999,"15min":1.5089999999999999},"min":{"1min":1.036,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.526,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.341},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.385,"15min":1.4870000000000001},"min":{"1min":1.034,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.4570000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.3700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31912,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.179,"15min":1.181},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5389999999999999,"5min":2.222,"15min":2.222},"last":1.026},{"interface":"front","average":{"1min":1.171,"5min":1.1319999999999999,"15min":1.1040000000000001},"min":{"1min":0.83399999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.335,"5min":1.601,"15min":1.601},"last":0.89900000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.359,"15min":1.4279999999999999},"min":{"1min":0.88,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.577,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.123},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.81299999999999994,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.627,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.365}]},{"osd":2,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.232,"15min":1.26},"min":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.512,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.131,"5min":1.224,"15min":1.254},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6040000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.331}]},{"osd":3,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2809999999999999,"15min":1.3979999999999999},"min":{"1min":0.81000000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.4630000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.2929999999999999,"15min":1.3280000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7390000000000001,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.0640000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.1699999999999999,"15min":1.2},"min":{"1min":0.74199999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5720000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.288,"15min":1.3280000000000001},"min":{"1min":0.89600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.514,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":1.2330000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.28,"15min":1.321},"min":{"1min":0.94099999999999995,"5min":0.755,"15min":0.755},"max":{"1min":1.804,"5min":1.887,"15min":1.887},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.2789999999999999,"15min":1.2729999999999999},"min":{"1min":0.82799999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.696,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.4530000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:54:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4330000000000001,"15min":1.4910000000000001},"min":{"1min":1.1160000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.649,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":1.413},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.419,"15min":1.464},"min":{"1min":1.0640000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.8520000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.5009999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.077,"15min":1.099},"min":{"1min":0.80100000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.151,"15min":1.2330000000000001},"min":{"1min":0.76400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5720000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.097}]},{"osd":1,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.86499999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.639,"5min":1.919,"15min":1.919},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1559999999999999,"15min":1.1930000000000001},"min":{"1min":0.874,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.4650000000000001,"5min":1.99,"15min":1.99},"last":1.0349999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.246,"15min":1.3089999999999999},"min":{"1min":0.92000000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.462,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.1639999999999999,"15min":1.218},"min":{"1min":0.83999999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.1819999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.319,"15min":1.4199999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.482,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.202},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3680000000000001,"15min":1.462},"min":{"1min":1.008,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":1.7070000000000001,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.244}]},{"osd":5,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.202,"15min":1.3149999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.7170000000000001,"5min":4.8099999999999996,"15min":4.8099999999999996},"last":1.419},{"interface":"front","average":{"1min":1.232,"5min":1.296,"15min":1.373},"min":{"1min":0.89800000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7430000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.99099999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3819999999999999,"15min":1.4930000000000001},"min":{"1min":0.96599999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7330000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.321,"5min":1.4330000000000001,"15min":1.579},"min":{"1min":0.73899999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.7410000000000001,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.3500000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.4259999999999999,"15min":1.528},"min":{"1min":1.0580000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.6930000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.385},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.49,"15min":1.6240000000000001},"min":{"1min":1.1100000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.79,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.151}]}]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:54:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.93500000000000005,"15min":0.997},"min":{"1min":0.58999999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.8560000000000001,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.087},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94599999999999995,"15min":0.97799999999999998},"min":{"1min":0.61899999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.413,"5min":1.661,"15min":1.661},"last":0.81000000000000005}]},{"osd":1,"last update":"Fri Sep 13 16:54:26 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.97899999999999998,"15min":1.0369999999999999},"min":{"1min":0.69499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.4259999999999999,"5min":1.931,"15min":1.931},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.1739999999999999},"min":{"1min":0.65900000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7490000000000001,"5min":1.849,"15min":1.849},"last":1.1819999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:54:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.21,"15min":1.3069999999999999},"min":{"1min":0.75800000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6519999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.256,"15min":1.3660000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4370000000000001,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.331}]},{"osd":4,"last update":"Fri Sep 13 16:54:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3069999999999999,"15min":1.423},"min":{"1min":0.84599999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7490000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.03},{"interface":"front","average":{"1min":1.329,"5min":1.3899999999999999,"15min":1.484},"min":{"1min":1.0880000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.8979999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.232}]},{"osd":5,"last update":"Fri Sep 13 16:54:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.4790000000000001,"15min":1.6020000000000001},"min":{"1min":1.1439999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.821,"5min":2.79,"15min":2.79},"last":1.383},{"interface":"front","average":{"1min":1.401,"5min":1.464,"15min":1.5940000000000001},"min":{"1min":1.0569999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.1640000000000001,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.5089999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:54:26 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5289999999999999,"15min":1.6579999999999999},"min":{"1min":0.97899999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.0049999999999999,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.444},{"interface":"front","average":{"1min":1.395,"5min":1.4590000000000001,"15min":1.5860000000000001},"min":{"1min":0.95799999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.948,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.21}]},{"osd":7,"last update":"Fri Sep 13 16:54:26 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.512,"15min":1.6379999999999999},"min":{"1min":0.99199999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8120000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.581,"15min":1.708},"min":{"1min":1.2470000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.9810000000000001,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":1.4750000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:54:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.91200000000000003,"15min":0.94399999999999995},"min":{"1min":0.53800000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.6919999999999999,"5min":1.925,"15min":1.925},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0740000000000001,"15min":1.117},"min":{"1min":0.83999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.6120000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.2889999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:54:30 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91700000000000004,"15min":0.95999999999999996},"min":{"1min":0.64700000000000002,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3560000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.871},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.016,"15min":1.0660000000000001},"min":{"1min":0.67200000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3819999999999999,"5min":2.028,"15min":2.028},"last":0.94599999999999995}]},{"osd":3,"last update":"Fri Sep 13 16:54:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2110000000000001,"15min":1.1930000000000001},"min":{"1min":0.64100000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.78,"5min":1.948,"15min":1.948},"last":1.129},{"interface":"front","average":{"1min":1.071,"5min":1.1279999999999999,"15min":1.1339999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.4530000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:54:30 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.28,"15min":1.3120000000000001},"min":{"1min":0.91100000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5940000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.355,"15min":1.4379999999999999},"min":{"1min":0.94999999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.6100000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.0600000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:54:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.46,"15min":1.554},"min":{"1min":1.0640000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6639999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.363,"15min":1.4079999999999999},"min":{"1min":0.97699999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.778,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.3169999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:54:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4830000000000001,"15min":1.5529999999999999},"min":{"1min":1.026,"5min":0.753,"15min":0.753},"max":{"1min":1.7190000000000001,"5min":2.625,"15min":2.625},"last":1.427},{"interface":"front","average":{"1min":1.357,"5min":1.468,"15min":1.5720000000000001},"min":{"1min":0.99299999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.792,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":1.337}]},{"osd":7,"last update":"Fri Sep 13 16:54:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.528,"15min":1.641},"min":{"1min":1.036,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8380000000000001,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5589999999999999,"15min":1.643},"min":{"1min":0.88800000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.7749999999999999,"5min":3.4060000000000001,"15min":3.4060000000000001},"last":1.573}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":370052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.7569999999999999,"5min":1.804,"15min":1.804},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.96899999999999997,"15min":0.99099999999999999},"min":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.212,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.83199999999999996}]},{"osd":2,"last update":"Fri Sep 13 16:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.91400000000000003,"15min":0.92300000000000004},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.456,"5min":1.954,"15min":1.954},"last":1.02},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0269999999999999,"15min":1.0649999999999999},"min":{"1min":0.68500000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3260000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:54:24 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2190000000000001,"15min":1.1830000000000001},"min":{"1min":0.879,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6279999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.153,"15min":1.1559999999999999},"min":{"1min":0.76400000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.6639999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.2110000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:54:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.254,"15min":1.304},"min":{"1min":0.89500000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6180000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.349,"15min":1.375},"min":{"1min":0.91700000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.698,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.278}]},{"osd":5,"last update":"Fri Sep 13 16:54:24 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4610000000000001,"15min":1.5149999999999999},"min":{"1min":0.98799999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.78,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.516},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.452,"15min":1.5149999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.704,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.4390000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:54:24 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.464,"15min":1.5229999999999999},"min":{"1min":1.0620000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.702,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.4139999999999999,"15min":1.4810000000000001},"min":{"1min":0.91000000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.7470000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:54:24 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4910000000000001,"15min":1.5409999999999999},"min":{"1min":1.167,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":1.8089999999999999,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.464},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.5349999999999999,"15min":1.5900000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":1.8720000000000001,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.3999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1404928,"data_stored":960420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88800000000000001,"15min":0.94099999999999995},"min":{"1min":0.56599999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.3200000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.01,"15min":1.028},"min":{"1min":0.64600000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.873}]},{"osd":2,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.01,"15min":1.036},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.997},{"interface":"front","average":{"1min":1.099,"5min":1.1160000000000001,"15min":1.1479999999999999},"min":{"1min":0.73099999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.7589999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":1.244}]},{"osd":3,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0149999999999999,"15min":1.0289999999999999},"min":{"1min":0.69499999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1950000000000001,"15min":1.2290000000000001},"min":{"1min":0.92500000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.534,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.0509999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3160000000000001,"15min":1.381},"min":{"1min":0.94599999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":3.0099999999999998,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":1.304},{"interface":"front","average":{"1min":1.319,"5min":1.3360000000000001,"15min":1.3580000000000001},"min":{"1min":1.054,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.6699999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.464}]},{"osd":5,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.417,"15min":1.488},"min":{"1min":0.98999999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":3.1110000000000002,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3899999999999999,"15min":1.46},"min":{"1min":0.97299999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.9279999999999999,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.5860000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4530000000000001,"15min":1.5309999999999999},"min":{"1min":1.044,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.6919999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.492,"15min":1.571},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.9870000000000001,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":1.696}]},{"osd":7,"last update":"Fri Sep 13 16:54:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.5269999999999999,"15min":1.6100000000000001},"min":{"1min":1.177,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.8889999999999998,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.629},{"interface":"front","average":{"1min":1.425,"5min":1.462,"15min":1.5269999999999999},"min":{"1min":1.0649999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.8300000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.5009999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:54:42.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T16:54:42.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.338+0000 7f7411e00700 1 Processor -- start 2024-09-13T16:54:42.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.338+0000 7f7411e00700 1 -- start start 2024-09-13T16:54:42.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c06cfb0 0x7f740c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f740c06c360 0x7f740c06c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0cedf0 con 0x7f740c160ef0 2024-09-13T16:54:42.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0cfa70 con 0x7f740c06c360 2024-09-13T16:54:42.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c06cd10 con 0x7f740c06cfb0 2024-09-13T16:54:42.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7410a00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f740c160ef0 0x7f740c0cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33452/0 (socket says 172.21.15.3:33452) 2024-09-13T16:54:42.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7410a00700 1 -- 172.21.15.3:0/2717254902 learned_addr learned my addr 172.21.15.3:0/2717254902 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-13T16:54:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411400700 1 --2- 172.21.15.3:0/2717254902 >> v2:172.21.15.157:3300/0 conn(0x7f740c06c360 0x7f740c06c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f740be00700 1 --2- 172.21.15.3:0/2717254902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c06cfb0 0x7f740c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f740b400700 1 -- 172.21.15.3:0/2717254902 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256430926 0 0) 0x7f740c0cedf0 con 0x7f740c160ef0 2024-09-13T16:54:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f740b400700 1 -- 172.21.15.3:0/2717254902 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f40035d0 con 0x7f740c160ef0 2024-09-13T16:54:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411400700 1 -- 172.21.15.3:0/2717254902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c06cfb0 msgr2=0x7f740c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411400700 1 --2- 172.21.15.3:0/2717254902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c06cfb0 0x7f740c06b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411400700 1 -- 172.21.15.3:0/2717254902 >> v1:172.21.15.3:6789/0 conn(0x7f740c160ef0 legacy=0x7f740c0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411400700 1 -- 172.21.15.3:0/2717254902 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0cf0d0 con 0x7f740c06c360 2024-09-13T16:54:42.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411400700 1 --2- 172.21.15.3:0/2717254902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 0x7f740c06c7d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7404009d90 tx=0x7f740400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915fc4130706c727 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:42.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f740b400700 1 -- 172.21.15.3:0/2717254902 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400e070 con 0x7f740c06c360 2024-09-13T16:54:42.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f740b400700 1 -- 172.21.15.3:0/2717254902 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400bb20 con 0x7f740c06c360 2024-09-13T16:54:42.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f740b400700 1 -- 172.21.15.3:0/2717254902 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404012430 con 0x7f740c06c360 2024-09-13T16:54:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 -- 172.21.15.3:0/2717254902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 msgr2=0x7f740c06c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 --2- 172.21.15.3:0/2717254902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 0x7f740c06c7d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7404009d90 tx=0x7f740400b330 comp rx=0 tx=0).stop 2024-09-13T16:54:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 -- 172.21.15.3:0/2717254902 shutdown_connections 2024-09-13T16:54:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 --2- 172.21.15.3:0/2717254902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 0x7f740c06c7d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 --2- 172.21.15.3:0/2717254902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c06cfb0 0x7f740c06b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.342+0000 7f7411e00700 1 -- 172.21.15.3:0/2717254902 >> 172.21.15.3:0/2717254902 conn(0x7f740c0721a0 msgr2=0x7f740c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.346+0000 7f7411e00700 1 -- 172.21.15.3:0/2717254902 shutdown_connections 2024-09-13T16:54:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.346+0000 7f7411e00700 1 -- 172.21.15.3:0/2717254902 wait complete. 2024-09-13T16:54:42.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.346+0000 7f7411e00700 1 Processor -- start 2024-09-13T16:54:42.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.346+0000 7f7411e00700 1 -- start start 2024-09-13T16:54:42.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 0x7f740c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f740c06cfb0 0x7f740c15a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411e00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c160ef0 0x7f740c163590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c15a8f0 con 0x7f740c06cfb0 2024-09-13T16:54:42.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c15abd0 con 0x7f740c06c360 2024-09-13T16:54:42.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411e00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c15aeb0 con 0x7f740c160ef0 2024-09-13T16:54:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f740c06cfb0 0x7f740c15a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f740c06cfb0 0x7f740c15a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40716/0 (socket says 172.21.15.3:40716) 2024-09-13T16:54:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 -- 172.21.15.3:0/2529975202 learned_addr learned my addr 172.21.15.3:0/2529975202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7410a00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 0x7f740c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411400700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c160ef0 0x7f740c163590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 -- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c160ef0 msgr2=0x7f740c163590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c160ef0 0x7f740c163590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 -- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 msgr2=0x7f740c158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 0x7f740c158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 -- 172.21.15.3:0/2529975202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404009a70 con 0x7f740c06cfb0 2024-09-13T16:54:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411400700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c160ef0 0x7f740c163590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:42.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f740be00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f740c06cfb0 0x7f740c15a070 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e9a0 tx=0x7f73f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7408c00700 1 -- 172.21.15.3:0/2529975202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8009720 con 0x7f740c06cfb0 2024-09-13T16:54:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7408c00700 1 -- 172.21.15.3:0/2529975202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f800c850 con 0x7f740c06cfb0 2024-09-13T16:54:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7408c00700 1 -- 172.21.15.3:0/2529975202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f800cc00 con 0x7f740c06cfb0 2024-09-13T16:54:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c15b190 con 0x7f740c06cfb0 2024-09-13T16:54:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.350+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c15a5b0 con 0x7f740c06cfb0 2024-09-13T16:54:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.354+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c06c7d0 con 0x7f740c06cfb0 2024-09-13T16:54:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.354+0000 7f7408c00700 1 -- 172.21.15.3:0/2529975202 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73f8013470 con 0x7f740c06cfb0 2024-09-13T16:54:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.354+0000 7f7408c00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73ec066730 0x7f73ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.358+0000 7f7408c00700 1 -- 172.21.15.3:0/2529975202 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f73f80840c0 con 0x7f740c06cfb0 2024-09-13T16:54:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.358+0000 7f7410a00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73ec066730 0x7f73ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.358+0000 7f7410a00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73ec066730 0x7f73ec068bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7400007fd0 tx=0x7f7400007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.362+0000 7f7408c00700 1 -- 172.21.15.3:0/2529975202 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f73f80515d0 con 0x7f740c06cfb0 2024-09-13T16:54:42.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.542+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f740c1643b0 con 0x7f740c06cfb0 2024-09-13T16:54:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.550+0000 7f7408c00700 1 -- 172.21.15.3:0/2529975202 <== mon.0 v2:172.21.15.3:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f73f80515d0 con 0x7f740c06cfb0 2024-09-13T16:54:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.550+0000 7f7408c00700 1 -- 172.21.15.3:0/2529975202 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7f73f8054bf0 con 0x7f740c06cfb0 2024-09-13T16:54:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73ec066730 msgr2=0x7f73ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73ec066730 0x7f73ec068bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7400007fd0 tx=0x7f7400007de0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f740c06cfb0 msgr2=0x7f740c15a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f740c06cfb0 0x7f740c15a070 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e9a0 tx=0x7f73f800ecb0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 shutdown_connections 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7f73ec066730 0x7f73ec068bf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c160ef0 0x7f740c163590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f740c06cfb0 0x7f740c15a070 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 --2- 172.21.15.3:0/2529975202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c06c360 0x7f740c158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 >> 172.21.15.3:0/2529975202 conn(0x7f740c0721a0 msgr2=0x7f740c0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 shutdown_connections 2024-09-13T16:54:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.554+0000 7f7411e00700 1 -- 172.21.15.3:0/2529975202 wait complete. 2024-09-13T16:54:42.581 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T16:54:42.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:42.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.746+0000 7fb6ffe00700 1 Processor -- start 2024-09-13T16:54:42.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.746+0000 7fb6ffe00700 1 -- start start 2024-09-13T16:54:42.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6ffe00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb70006ccf0 0x7fb70015d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6ffe00700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb700070280 con 0x7fb70015ac30 2024-09-13T16:54:42.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6ffe00700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70015dc00 con 0x7fb700159840 2024-09-13T16:54:42.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6ffe00700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70015dee0 con 0x7fb70006ccf0 2024-09-13T16:54:42.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6f6000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb70006ccf0 0x7fb70015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6f6000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb70006ccf0 0x7fb70015d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40730/0 (socket says 172.21.15.3:40730) 2024-09-13T16:54:42.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6f6000700 1 -- 172.21.15.3:0/1850412222 learned_addr learned my addr 172.21.15.3:0/1850412222 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:42.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6ff400700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb70015ac30 0x7fb7000b2fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33458/0 (socket says 172.21.15.3:33458) 2024-09-13T16:54:42.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6f6000700 1 -- 172.21.15.3:0/1850412222 >> v1:172.21.15.157:6789/0 conn(0x7fb700159840 legacy=0x7fb70006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6f6000700 1 -- 172.21.15.3:0/1850412222 >> v1:172.21.15.3:6789/0 conn(0x7fb70015ac30 legacy=0x7fb7000b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6f6000700 1 -- 172.21.15.3:0/1850412222 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70015e1c0 con 0x7fb70006ccf0 2024-09-13T16:54:42.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6fe000700 1 -- 172.21.15.3:0/1850412222 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412199166 0 0) 0x7fb700070280 con 0x7fb70015ac30 2024-09-13T16:54:42.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6f6000700 1 --2- 172.21.15.3:0/1850412222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb70006ccf0 0x7fb70015d620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009910 tx=0x7fb6e8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c616691eba5b126 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:42.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6fe000700 1 -- 172.21.15.3:0/1850412222 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800ba60 con 0x7fb70006ccf0 2024-09-13T16:54:42.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6fe000700 1 -- 172.21.15.3:0/1850412222 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb6e800bbc0 con 0x7fb70006ccf0 2024-09-13T16:54:42.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.750+0000 7fb6fe000700 1 -- 172.21.15.3:0/1850412222 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8009e60 con 0x7fb70006ccf0 2024-09-13T16:54:42.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.754+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1850412222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb70006ccf0 msgr2=0x7fb70015d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.754+0000 7fb6ffe00700 1 --2- 172.21.15.3:0/1850412222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb70006ccf0 0x7fb70015d620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009910 tx=0x7fb6e8009c20 comp rx=0 tx=0).stop 2024-09-13T16:54:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.754+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1850412222 shutdown_connections 2024-09-13T16:54:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.754+0000 7fb6ffe00700 1 --2- 172.21.15.3:0/1850412222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb70006ccf0 0x7fb70015d620 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.754+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1850412222 >> 172.21.15.3:0/1850412222 conn(0x7fb7000b0cf0 msgr2=0x7fb7000b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.754+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1850412222 shutdown_connections 2024-09-13T16:54:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.754+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1850412222 wait complete. 2024-09-13T16:54:42.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.754+0000 7fb6ffe00700 1 Processor -- start 2024-09-13T16:54:42.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6ffe00700 1 -- start start 2024-09-13T16:54:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6ffe00700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70006ccf0 0x7fb70013d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6ffe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb700159840 0x7fb7000c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6ffe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70015ac30 0x7fb70013dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6ffe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70013d9a0 con 0x7fb700159840 2024-09-13T16:54:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6ffe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000c8490 con 0x7fb70015ac30 2024-09-13T16:54:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6ffe00700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000c86c0 con 0x7fb70006ccf0 2024-09-13T16:54:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb700159840 0x7fb7000c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb700159840 0x7fb7000c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40738/0 (socket says 172.21.15.3:40738) 2024-09-13T16:54:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 -- 172.21.15.3:0/1474734378 learned_addr learned my addr 172.21.15.3:0/1474734378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-13T16:54:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6ff400700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70015ac30 0x7fb70013dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6fea00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70006ccf0 0x7fb70013d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 -- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70006ccf0 msgr2=0x7fb70013d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70006ccf0 0x7fb70013d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 -- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70015ac30 msgr2=0x7fb70013dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:42.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70015ac30 0x7fb70013dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:42.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 -- 172.21.15.3:0/1474734378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e8009570 con 0x7fb700159840 2024-09-13T16:54:42.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.758+0000 7fb6f6000700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb700159840 0x7fb7000c7f50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8000c00 tx=0x7fb6e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:42.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.762+0000 7fb6f7400700 1 -- 172.21.15.3:0/1474734378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8011a70 con 0x7fb700159840 2024-09-13T16:54:42.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.762+0000 7fb6f7400700 1 -- 172.21.15.3:0/1474734378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb6e801f070 con 0x7fb700159840 2024-09-13T16:54:42.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.762+0000 7fb6f7400700 1 -- 172.21.15.3:0/1474734378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8011df0 con 0x7fb700159840 2024-09-13T16:54:42.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.762+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb700152130 con 0x7fb700159840 2024-09-13T16:54:42.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.762+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7001526f0 con 0x7fb700159840 2024-09-13T16:54:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.762+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70014dfb0 con 0x7fb700159840 2024-09-13T16:54:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.766+0000 7fb6f7400700 1 -- 172.21.15.3:0/1474734378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6e8026480 con 0x7fb700159840 2024-09-13T16:54:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.766+0000 7fb6f7400700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb6dc066730 0x7fb6dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.766+0000 7fb6f7400700 1 -- 172.21.15.3:0/1474734378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fb6e8021070 con 0x7fb700159840 2024-09-13T16:54:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.766+0000 7fb6fea00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb6dc066730 0x7fb6dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:42.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.766+0000 7fb6fea00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb6dc066730 0x7fb6dc068bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0000f60 tx=0x7fb6f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:42.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:42.774+0000 7fb6f7400700 1 -- 172.21.15.3:0/1474734378 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb6e8053000 con 0x7fb700159840 2024-09-13T16:54:43.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.010+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb70015d620 con 0x7fb700159840 2024-09-13T16:54:43.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.010+0000 7fb6f7400700 1 -- 172.21.15.3:0/1474734378 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fb6e802b070 con 0x7fb700159840 2024-09-13T16:54:43.015 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-13T16:54:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1108978187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1108978187},{"type":"v1","addr":"172.21.15.3:6833","nonce":1108978187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2401450768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2401450768},{"type":"v1","addr":"172.21.15.157:6836","nonce":2401450768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2879546209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2879546209},{"type":"v1","addr":"172.21.15.157:6837","nonce":2879546209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3890157272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3890157272},{"type":"v1","addr":"172.21.15.3:6835","nonce":3890157272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[]} 2024-09-13T16:54:43.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb6dc066730 msgr2=0x7fb6dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:43.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb6dc066730 0x7fb6dc068bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0000f60 tx=0x7fb6f0009040 comp rx=0 tx=0).stop 2024-09-13T16:54:43.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb700159840 msgr2=0x7fb7000c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:43.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb700159840 0x7fb7000c7f50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8000c00 tx=0x7fb6e8004770 comp rx=0 tx=0).stop 2024-09-13T16:54:43.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 shutdown_connections 2024-09-13T16:54:43.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:6832/3433838649,v1:172.21.15.157:6833/3433838649] conn(0x7fb6dc066730 0x7fb6dc068bf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:43.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70015ac30 0x7fb70013dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:43.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb700159840 0x7fb7000c7f50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:43.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 --2- 172.21.15.3:0/1474734378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70006ccf0 0x7fb70013d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:43.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 >> 172.21.15.3:0/1474734378 conn(0x7fb7000b0cf0 msgr2=0x7fb700159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:43.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 shutdown_connections 2024-09-13T16:54:43.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-13T16:54:43.014+0000 7fb6ffe00700 1 -- 172.21.15.3:0/1474734378 wait complete. 2024-09-13T16:54:43.020 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 40 2024-09-13T16:54:43.044 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-13T16:54:43.044 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T16:54:43.044 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:54:43.044 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:43.066 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:54:43.066 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T16:54:43.067 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:43.129 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T16:54:43.130 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:54:43.130 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:43.156 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:54:43.156 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:54:43.156 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:43.223 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:54:43.224 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T16:54:43.224 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T16:54:43.224 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:43.304 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T16:54:43.304 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T16:54:43.304 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:43.375 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T16:54:43.375 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T16:54:43.376 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:46.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-13T16:54:49.477 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T16:54:49.477 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T16:54:49.477 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:52.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T16:54:52.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-13T16:54:55.579 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T16:54:55.579 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T16:54:55.579 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:55.661 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T16:54:55.661 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T16:54:55.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:58.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-13T16:54:58.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T16:54:59.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:54:59.163 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:01.762 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T16:55:01.763 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T16:55:01.763 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:55:04.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-13T16:55:04.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-13T16:55:07.865 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T16:55:07.865 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T16:55:07.865 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:55:07.952 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T16:55:07.952 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T16:55:07.952 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T16:55:07.952 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:55:08.038 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T16:55:08.038 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T16:55:08.038 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:55:08.107 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T16:55:08.107 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T16:55:08.107 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T16:55:08.108 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:55:08.145 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T16:55:08.145 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T16:55:08.145 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:55:08.177 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T16:55:08.177 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T16:55:08.177 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:55:08.253 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T16:55:08.254 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T16:55:08.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-13T16:55:08.308 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:55:08.308 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T16:55:08.618 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:55:08.618 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T16:55:08.772 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:55:08.772 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T16:55:08.946 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:55:08.946 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T16:55:09.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-13T16:55:09.134 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:55:09.135 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T16:55:09.384 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:55:09.384 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T16:55:09.605 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:55:09.605 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T16:55:09.827 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:55:09.827 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T16:55:10.035 INFO:tasks.ceph:Archiving mon data... 2024-09-13T16:55:10.036 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/data/mon.a.tgz 2024-09-13T16:55:10.037 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-13T16:55:10.056 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.QzIkidJCGl 2024-09-13T16:55:10.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QzIkidJCGl 2024-09-13T16:55:10.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.QzIkidJCGl 2024-09-13T16:55:10.333 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.QzIkidJCGl is 388KB 2024-09-13T16:55:10.407 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.QzIkidJCGl 2024-09-13T16:55:10.423 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/data/mon.b.tgz 2024-09-13T16:55:10.423 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-09-13T16:55:10.442 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.6uJrp5GrTr 2024-09-13T16:55:10.442 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6uJrp5GrTr 2024-09-13T16:55:10.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.6uJrp5GrTr 2024-09-13T16:55:10.724 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.6uJrp5GrTr is 511KB 2024-09-13T16:55:10.799 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.6uJrp5GrTr 2024-09-13T16:55:10.819 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/data/mon.c.tgz 2024-09-13T16:55:10.819 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-09-13T16:55:10.878 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.MHNBgT9SJD 2024-09-13T16:55:10.878 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.MHNBgT9SJD 2024-09-13T16:55:11.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.MHNBgT9SJD 2024-09-13T16:55:11.170 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.MHNBgT9SJD is 510KB 2024-09-13T16:55:11.247 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.MHNBgT9SJD 2024-09-13T16:55:11.264 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T16:55:11.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T16:55:11.270 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T16:55:11.339 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T16:55:11.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T16:55:11.345 DEBUG:teuthology.orchestra.run.smithi157:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T16:55:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T16:55:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T16:55:11.446 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T16:55:11.448 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/remote/smithi003/crash 2024-09-13T16:55:11.449 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T16:55:11.483 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/remote/smithi157/crash 2024-09-13T16:55:11.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T16:55:11.516 INFO:tasks.ceph:Shutting down logrotate 2024-09-13T16:55:11.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:11.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:11.622 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:11.651 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:11.683 INFO:tasks.ceph:Compressing logs... 2024-09-13T16:55:11.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:55:11.701 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:55:16.037 INFO:tasks.ceph:Archiving logs... 2024-09-13T16:55:16.037 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/remote/smithi003/log 2024-09-13T16:55:16.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T16:55:16.784 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/remote/smithi157/log 2024-09-13T16:55:16.785 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T16:55:17.281 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T16:55:17.290 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T16:55:17.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T16:55:17.297 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T16:55:17.427 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T16:55:17.437 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T16:55:17.437 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:55:17.443 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:55:17.478 INFO:teuthology.orchestra.run.smithi003.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T16:55:17.478 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================== 2024-09-13T16:55:17.478 INFO:teuthology.orchestra.run.smithi003.stdout:+hv01.front.sepi 67.205.162.81 3 u 35 64 377 0.097 0.534 0.397 2024-09-13T16:55:17.479 INFO:teuthology.orchestra.run.smithi003.stdout:*hv02.front.sepi 63.231.80.2 3 u 41 64 377 0.122 0.481 0.400 2024-09-13T16:55:17.479 INFO:teuthology.orchestra.run.smithi003.stdout:+hv03.front.sepi 74.6.168.72 3 u 35 64 377 0.122 1.482 0.382 2024-09-13T16:55:17.479 INFO:teuthology.orchestra.run.smithi003.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:55:17.484 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T16:55:17.484 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2024-09-13T16:55:17.484 INFO:teuthology.orchestra.run.smithi157.stdout:*hv01.front.sepi 67.205.162.81 3 u 38 64 377 0.104 -0.127 1.080 2024-09-13T16:55:17.485 INFO:teuthology.orchestra.run.smithi157.stdout: hv02.front.sepi 63.231.80.2 3 u 36 64 377 0.119 0.670 0.679 2024-09-13T16:55:17.485 INFO:teuthology.orchestra.run.smithi157.stdout: hv03.front.sepi 74.6.168.72 3 u 34 64 377 0.104 0.734 1.036 2024-09-13T16:55:17.485 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T16:55:17.486 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T16:55:17.494 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T16:55:17.495 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T16:55:17.543 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T16:55:17.551 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T16:55:17.559 INFO:teuthology.task.internal:Duration was 1012.613108 seconds 2024-09-13T16:55:17.560 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T16:55:17.569 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T16:55:17.569 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T16:55:17.575 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T16:55:17.728 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T16:55:17.729 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-09-13T16:55:17.729 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T16:55:17.765 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T16:55:17.765 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T16:55:17.799 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T16:55:17.799 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:55:17.817 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:55:17.910 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T16:55:17.910 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T16:55:17.916 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T16:55:18.030 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T16:55:18.040 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T16:55:18.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T16:55:18.074 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T16:55:18.108 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T16:55:18.118 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:55:18.124 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:55:18.153 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-09-13T16:55:18.186 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-09-13T16:55:18.215 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:55:18.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:55:18.247 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:55:18.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:55:18.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T16:55:18.286 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T16:55:18.286 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/remote/smithi003 2024-09-13T16:55:18.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T16:55:18.336 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903832/remote/smithi157 2024-09-13T16:55:18.336 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T16:55:18.382 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T16:55:18.382 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T16:55:18.388 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T16:55:18.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T16:55:18.451 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T16:55:18.452 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T16:55:18.493 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T16:55:18.493 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T16:55:18.499 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T16:55:18.515 INFO:teuthology.orchestra.run.smithi003.stdout: 1181010 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 16:55 /home/ubuntu/cephtest 2024-09-13T16:55:18.521 INFO:teuthology.orchestra.run.smithi157.stdout: 1181012 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 16:55 /home/ubuntu/cephtest 2024-09-13T16:55:18.526 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T16:55:18.543 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T16:55:18.647 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable} duration: 1012.6131076812744 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T16:55:18.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T16:55:18.754 INFO:teuthology.run:pass